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ABSTRACT r ' ' ' - - - 

This final report describing the operation and ' - 

research, of the EISO project, which provides online bibliographic 
search services to educators of Ontario, briefly suaaarizes the first 
and second interim reports. In addition, it discusses ^u^ers and the 
uses of EISO i^nformation by providing mini-case studies to illustrate 
quantitative data and identifying factors contributing t<> use or 
non-use of search results: The monthly and yearly satisfaction levels 
of users who ^d and did- not utilizp the services of | .~ 
intermediaries — Educational Information consultants-- are alsb 
explored. The negotiation process bet.ween- user and int^mediary is 
analyzed and a preliminary model.. of the process is developed. A 
systems evaliiation of user satisfactibii employing socibiogical and 
process variables is also included. Recejtt trends and :ilevelopments in 
online searching are reviewed to illustrate how sthese itrends and ' 
ElSO's own researcfr" and experience*have resulted d,n the -pro ject« s 
operation. Appendices include user evaluation questionnaires, EliSO , „ 
order forms, . and publicity brochures; (Author/MBR) |: ' 
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This- volume , is the 'thi-rd and final report dies crib ing" the 
ope'ration ancr^ research of the Educational Information Sys,tem . 
for Ontario project funded by the Ontario Minis^try of Educa- 
*tion in March 19 75' to prpvide online "^ibliograplj^c search 
services -to the educators of this province * 



This report l)e.gins by ^.briefly siimmarizi^ig the f i::rs.t\ and 
•second Interim Reports^ that 'described the scope of the teip/^ 
search, the conceptual framework^^ data collection tecltniques, 
the imi)lementation of the servdcCju^and preliminary evaluation 
results. It then devotes a chapter to the users and the uses 
of EISO information providing mini-case studies to ^f lesh out 
the quantitative, data and identifying factors contribu-f-lng to. 
the use or non-use. of search results^ Chapter 3 pr-eser.: 
monthly and yearly satisfaction' averages ♦of" EISO users and . 
^mpares satisfaction levels of users J^ho did arid" did not go 
through inte.rmediaries or Educational Information Consultants 
The negotiation p-roces's between the user and , the seai^ch ana- 
lys^t], in analyzed in Chapter 4. an(d -a preliminary model of the 
pf4$^<ei:ss . developed. Chapter ,5 presents a systems evaluation < 
of liser satisfaction employing sociological and ppocess vari- 
ablesv The concluding chapter reviews recent t'rends and ; ^ " 
developments in online searching and shows how these ^^^d the 
fiiidings of our; own research and experience have resjalt^ed in 
the op er at ion. of EISO as it is carried on in 1978. 
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CHAPTER 1 
INTRODUCTION . '. 

c ^ . - /■ 

This volume is the third and finat report .of- the Educational . 
Information System for Ontario, a research contract funded in 
March 1975 'by the Ministry of Education Cpf the Province of 
Ontario. Though much of the research compohent of the project 
is now completed the service aspect of EISO continues to pro- 
vide Ontario ed jr^^ors with, online bibliogra|)hic retrieval of 
refere^ees^ in eu^^atien and^^it-he social sciences" and, with " 
.access to original documents/ . 

BaoRgraund .V * - . 

EISO was /funded to develop, evaluate, and analyze an 
information dissemination system ba:.cd upon computerized 
jretrieval of bibliographies. It diffe sd from the usual 
research contract in that^it was not only to provide a 
rigorous research study firmly grounded in theoretical con*—-' 
cepts, but it was also to have, a' largely developmental com*-'\ 
ponent in the implementation of a search service available to 
educar&ors on a .f^-paying Jisis. Thus,^not only were normal 
aspects of contract research -uch as research design, data 
collection and analysis required, but. also some more unusual 
activities. * Among- these were the^ creation- of a business 
system, publicit:!ff materials, promotional workshops, and a ^ ' 
program to train Educational Information Consultants who were 
to act as information intermediaries or linkage agents in 
geographically remote "areas. Professional development acti- 
vities, seminars,' and demonstrations added still other aspects 



to zhe study. The aspe.ct^ of * th'e contract that was- most readily 
apparent to educators and^ the one that still cont;inues was, 
however, the^ actloal creation of a fully operatioaai information 
service offfering an' interactive • online bibliographic search 
service to the'^^ducat ors of Ontario by providing them access 
to existing data bases such as the Ontario Educational Research 
Information System CONTERTS) , BRlt*, Psy chotogical Abstr*aci^ , 
ExceptionaZ Chi^ld.^Eduoat'ion Abstraats^ SociaZ Sciences Cita^ 

jti.oH .''Index ^...DrL.s ser.ta^t i.o n . Ab e.t v^.a.otQ. , _ln'tern at iondl _ and_ .other , 

data bases re.lev.ant to educators- 

The EISO Service 

•* / 

To use the EISO search serviqe, the educator jsiubmits his 
information request to the search analyst, a specially trained 
reference librarian, who develops a searclj strategy that is 
machine- readable . Bibliographies are nxxn;ted offlirie'by a 
major commercial supplier like Locklr*red or System Development 
Corporation and mailed to the user who may .then order original 
documents in either microfiche or .paper copy ' formats • A fee 
based on the amount of professional and computer- connect time 
used'iSu charged for the search and .a flat rate for duplicates 
of original items. 



T^ie^ 



Interim Reports - ' ^ . 

ve ^ir>Bt Inter^im Report (Ausjter' and'LaVton, 1976) 
covered the j^eriod March 1975 to February 1976, Divided into 
eleven • chapters,, it recounted the, creation o-f EISO, both its 
service ^n^ research components. It reviewed the literature 
on information centres^ intermediaries, and change in educa- 
tion, described the problems encountered in introducing a new 
s*ervice* into'^an established organizational environment, ^9Ut- 
lined the initial training program for the Educational Iijifor-': 
matioif Consultants, and- outlined the design and operation of ' 
the business ^system that EISO' -was to follow. That First 
Interim Report also presented the researcl^ design that was 
used to monitor and evaluate EISO's operation and the satisrr 
faction of its users. A systems, model u'sing sociological. 



psycho-logdcad^ giid economic variables within :an ^adop$rion o£, 
^innovations f^ramevrorl? was dev^eloped aTtdy^^^X^mXriB^Ty data col- 
lection instruments were* designed.^ La^t,. some ^oaxlly telitative^ 
reseaxjcii findings were pre sent e'd ' - ^ - " ' 
. ' El SQ^'s Second Iri-tez'^m Repo2*t CLa^^zh1^'BTid Au^^ 

.'consisted o£ sel^en chapters cpveri^ng the period- March < 19 7€ to , • 
*^ ' / ^ ' - ' V - ■ ^ . . 'V 

F.^bruary- 1-9^7 • ' With- the i^ese^rch design complreired and tjie -^z 

""bperation. o.£ th-e service \^Xl qnder way , ,th^ -setond .yea-r * s ' . 

r.qport 'focused on. the -clients '-of ISO : their geographical- 

" -T ^ , • . - . "t 

'location, 'employing prgani z^tion^ pe.rsonal .^charac^^ 

~ , . ' , • , ■ * ^ *. "1 • * < "* " 

and most important, the'ir level^s^ o^^t^^?atf$f action^-^^rxth^ EPSO . 

Reliability o^^ s cales and - items us^^i^fibS coll^ Cbnceining 

^usex' satisfaction was - def axled as were tijie xr^ijSas, social 

correlates, and intervening variables ' of user satisfaction. . ^ 

Mini-case studies showed the "impact" that EISO had on the . 

director, codrdinktor prrdncipal, and teacher -of a school ; 

board. tn 'addiction, the economics, of opierating EISO werQ ^ 

considered, other data^^base search .services in Canada were 

identified,* aVid 'ali:ernative organizational structures for 

EISO, in the future were posed.* Both the first and second 

reports included' 'extens ive appendices of all data collection 

instruments ("and forms used in -the research as well as th^ 

service aspects of the contract. ^\ 



_ . Final Report- 

Thi^s third and final report pre.sents cumulative satis- 
faction data collected over a three-year ^period . It also 
delves into the uses of information obtained through EISO 
and . examines the negotiation process between the'search 
-analyst and th-e user whe'n a request is placed. A picture of 
EISO as it exists in mid- 1^78 concludes the volume. Appen- 
d'ic^s illustrate new business and monitoring- forms- develope^d 
that reflect 'the transition and changed emphasis from research 
p.roject to service provision. 



. CHAPTER 2 : \ 

/THE -USER^ AND THE .^SBS*' OF 'EISo" INFORMATION 



r 



'The gr'owing importacnce of the p"rovision\ of information by . ' 
automated means, has been reflected in the increasing n timber 
o£ articles and studies' devoted to online reference services. 
The focus of much of ^this wbrlfc centres aroun.dj developing"" • 
'accurate costing techniques [Cooper and De Wath, 19770 and.' 
describing and- evaluating sp'ecific -services . (FiiTSchein. and 
^Summit, 1977, and Lancaster, 1969). The users of these ' 
services have received little attention with the' notable 
exception of-Wanger's recent survey • C19"7 6) . Generally;,' how- 
^v^r, concern for the user and the way in which he uses the 
'information he^ receives has been of greater concern to educa"- 
tional researchers and sociologists than information special- 
ists (Hood, 1978, and Rittenhousey 1971). This concentration 
is understandable. « Information managers, librarians^ and 
■search analysts can rarely influence the' use of the informa- 
tioh they provide to the use"f: local exigencies, administrator 
commitment, financial constraints, the political climate may 
all- impinge on the ultimate use of retrieval information.. 
Tronically, however, it is the effective use and impact of ■ tnSS^ ' 
search results that may determine whether or not a u'^er per- 
chives the service to be useful and' becomes a satisfied client 
who returns to the seai^ch service to place further reques-ts. » 
And it is upon the Hjallding" up of such a repeat-user, clientele 
that the survival' . of a search service may depend. 

The. purposes-; of this chapter, therefore, are . to identify 
EISO users oyer the past three years, present their reasons for 
sjeeking information, provide mini-case studies of how the 



information tfiey received , impacted upoii their'local Situations, 

-* * • • ' ■ . • • . * 

and identify factors *that coiitributed to the. use or; non-use of^ 



information received. 



User Characteristics i * 

The eonce^tuial , framework and method fori collecting -data 
on EISO users vv^"^re • described in. the fi2?s^: Intez^im Repor'ti 
Maa:ch l9 75-Febfuary 1976 CAuster and Lawtonf; 1976) and the^ 
Second Inte-rim Repo^'-bi March 19 76 -February 1977 (Lawton and 
Auster, 1977) . Briefly, the *user was vi,ewed within an adop- 
tion of innovation model using sociological , 'psycholog^ical 
and economic perspectives. Each, user/was sent User Evalua- 
tion Questionnaire (Appendix A) six weeks after placing a 
search* Additional data weqjfe collected^^ by the search analyst 
on a Search Request Form (Appendix ^) , and on an EISO Service 
Evaluation^ Data Sheet (Appendix G) . ^'Finally, a sample of 
users was interviewed using an/Interview Schedule . (Appendix D) . 
^ It will be noted that data 'for setond-year sfearche's- presented 
below now include 371 returned evaluation. qXies tionnaires 
covering* the peri-6d April 1, 1976 to March "-'5a, 1977. Third ^ 
year data cover the period April 1, 1977 to December- 31, 1977. ^ 

Location > . ' 

As in the past two years, the majority of EISO's most 
recent users come from the Metropolitan- Toronto area, providing 
the mainstay of the service (Table 1). The fact that this high 
figure has remained fairly stable is ,all.the more interes't^ing 
because some professional libraries ^in Idscal schoo;i boards 
have recently purchased -their own terminals and are now 
offering their own search services. This leads one to suspect 
that the actual number of users from the greater Toronto area 
may be going -up but that this x-ncrease is being partially 
obscuredand absorbed^by other available services . V 

The increased number of • users from the^j Ottawa area^(1.9%.r 
to 7. 8%) may^ reflect the fact that several workshops and '^^eminars 
were given to governmental officials and a'^t summer courses^^ifiJeld - 
tliere thus raising awareness of the service and acting as a 
stimulant for usage. ' / . ^ ^ - 



TABLE 1 

LOCATION OF USER'S. ORGANIZATION 



Loca^ 



ion 



Yeair 1 . 
Cn = 386) 



Percent 

" If 



Year 2 
Cn = 371-r 



Yeax. 3 
Cn = 129> 



District: 



■ 14.8 



' 6.7 



4.7 



County or Regional 
Government 



23.-3 



25.3 



:i 27.9 



Mstrcjpplitan Toronto <^ 49.5 



55.8 



\ 53.5 



Ottawa 



London 



l.-O 



4.^ 



1.9 



7k S 



0.0 



Piamilton 



0.5 



1.3 



0.0 



Windsor 



2.6 



2-2 



1.6 



Suidbury 



1.3 



■ 3-2 



0.0 



Canada Cnot, Ontario) 



1-0 



0.8 



3.9 



Not Canada 



1-0 



1.3 



0.8 
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The ris,e o£ the number df^ users 'from proyinces dther 
than On-tario to 3..9%) nf^y reflect El SO » s n^w ' p o 1 i cy/. "of 

processing all req'uests. received regardless o£ thg.ir geogra- 
phical origin. , - ^ , ^ " " 

Pers^onal Charax:£erist ics ' \ 



* The age., s.ex, education, and^ professional /activxtie^ of 

EIBO users^ have remained^^stable over the three ye'ars the - 

service h^s been in operation^' Two- thirds of the uSers C9n- / 

tifiue to be male (Table 2J , ^nd over three-quarters of all ' 

users^ are between 26 and 45 years^ of age [Table 3). Most 

users have occti^ied their j^resent position for less than two 

years enable 4) . Over one-half (Tab^e 5) are officers, of-a 

professional organization,' have participated in an educational 

research project in the past 'five years (Table ,^), and" have 

•presented a .professional' paper during that time (Table 7^. 

Six:ty-n.ine percent "have earned graduate degrees (Tab\e 8) whil 

57% are^. currently enrolled in, or a^e planning to apply fox,*^ . 

academic programs leading to ^a higher' degree (Talkie 9) . _ \\ 

.■''*'* ' * . 

' ^ /^^ Clearly, EISO usexLS are a highly professional, dynamic, 

well- educated, and aSibitiott^ group 

Organization^ ^ ' ' ^ , - ^ , 

While OISE i^s still the institution with which most 
users are affiliated, it. provides only a little over one-third 
of tHe total number of users (36.6%), a decline from 39*5%%the 
previous year. Faculties of^ education , Ministry, of /Education , 
and Other government agencies continue to provide- increasingly 
more users while the; number of. school board clientele remain 
steady (Tab^le 10). 



Role . d 

The roles performed by EISO usefs .remained for the most 
part unchanged. Table 11 confirms that administration and 
teaching are^ still the most important roles occupied: by EISO . 
users. These are^ followed by graduate *work and research. 
These profiles of user roles'" have shown little vari^ation over 
a three-year period and will probably continue to remain stable 



• TABLE t 
SEX OB EISO USERS 

» 

r =~ — — 



Sex 



-f 



. tear 1 . - 
L . Cn =,376)' 



Percenl: 

! I 



Year] 2_ 
Cn = 370) 



Year 3 . 
Cn = 129) 



■ V •' ■ '- 

: ^Female, 



69.7 



30.3 



63.5 
36.5 



68.2 
31- ff 



■ / . 








■ / • '. 






\ ^ 


^ - -"■■! • ■ 


TABLE 3 
AGES OF "EISO US 

1 


ERS ' 


• ■ ■ 






Percent 




5 Age Range- 
in Years 








> Year 1 
j;^= 183) 


Year 2 
Cn = 202) 


• Year 3 
.Cn-= 103) 


25 or under 


2.2 


3.5 ' ■• 


1.9 


26 to 35^ 


-49.7 ' ■■ 


36.7 


- 40.8 - , ^ 


36' to 45 


" 36.1 ' / . 


41-6 


38.8 


46 to 55 


' ■ ■ 9.3 ■ . ■ 


15^8 


14.6 


56 to 65 . 


.2.7 : / 


0.5 


3.9 ■ 


over 65 


0.0 / / 


l.d; 


0.0 


... ." . " ' ■ ■ t . .. , 
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8 



^. 18. 



1 . . « „ . TABLE 4, - , , 
NUMBER JOB YEARS IN. CURRENT POSITION' 



o Percent 



Interval in Years *^ 



Year 1 
Cn =>18.4> 



-Year 2 
Cn = 203) 



Year 3 
tn = itei)' 



Less 
1 



2 
3 

4 to 6 



10.9 
^ 9.8 

- 24.5 



13.3 \ 
11.3 ' 

15.3^<" 

/- 

11-3 V 
21.7 



1^.4' 
Is. 5 

14.6^ 
6.8\ 
15.5 















7 to 9 


, . 14. 1 - i ' 


14.8 


8.7 




10 or more 

T 


8.2 . 1 " 

■ . 1 . 


12.3 


20.4 






! 






» 




\ 

I .1 . 


i 


V 






TABLE 5 
OFFICER OF 'a PROFESSIONAL 


ORGANIZATION 

. -ikL 










Percent 






Respaase 


Year ^' ^ . 
Cn = 183) 


. Year 2 
Cn = 202) 


Year 3 
Cn = 103) 






,61.7 


58.4 


59.2 


f 


•No 


38.3 


41.6* 




. 40.8^ 
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TABLE 6 



PAFITICIPATED IN" EDUGATICmL RESEARCH PROJECT 
IN TFE PAST FIVE YEAi?g. 



,i ' Response 



Percent 



Year 1 

." > Cn =-185.) 



Year 2- 
Cn = 200) 



Year 3- 
Cn ==r99) 



Yes- 



75. 7^: 



74.5 



63.6 



No 



"24.3 



'25.5 



36.4 ^ 



~ ■ ' -TABLE' T ' - 

PRESEJJTED A PROFESSICMAL.. PAPER IN THE 
PAST FR^ YEARS ' 



^ Response 



Year 1 
Cn =. 185) 



Percent 



Year 2 
Cn = 200)-^ 



Year 3. 
Cn = 103) 



Yes 



53.1 



56.0 



51.5 



No 



47.0 



44.0 



HTGHEST DEGREE EAKNED 



Degr^ 



Percent 



Year 1 
Cn =, 175y 



Year 2 
CnA= 19^) 



Year 3 
Cn -=f 100). 



Badielor^s 



Master *s ^ 

■ o 



Doctorate . 



other 



30^ 
54.3 
'l4.9 
0.6 



28.4, 
56.7 
14.9 
0-0 



31.0 
50.0 
19.0 

^.0 



. - ' - f TABLE 9 ' 

PRESENT OR PLAI^D STUDIES FOR HIGHER DEGREE 



Response 



Percent 



Year 1 \ 
Cn = 382) ' 



Year 2 
. Cn •= 200) 



Year 3 ' 
Cn = 102) 



Yes 

Not Sure 
No 



37.2 



20.7 



42.1 



45.5 



<fr ,16.5 



38.0 



51.0 



10.8 



38-2 



■'TABLE 10 
ORGANIZATION OF PRITIARY ASSOCIATION 



Organization 



Percent 



Year 1 
Cn = 380) 



Year 2 
(n = 369) 



Year -6. : 



Public Board 



Separate Board 



34". 2 



10.5 



.44.7 



24.4 



5-.1 



29.5 



30 ..2 : 



1.6 I 



31.8 



Private School 



D.8 



0.8 



0.0 



CAAT 



Faculty of Ediicatioii 



4-7 
2.1 



4.1 



7.3 



2.3 



OISE 



50.8 



39.5 



• 36.6 



Uhiversity 



6.1 



6.5 



4.7 



Ministry o£ Education 



3.7 



5.4 



7.0 



Professional Associations 01.8 



3.8 



2.3 



Government 



1.8 



1.1 



3.1 



Business 



1.8 



0.5 



1.6 



-bther 



2.6 



1.6 



2.3 





* TABLE 11" ^ ' / 






'*^PRIMAKY PROFESSIONAL ROLE 




Role 


Percent 




- Year 1 ^ Yeax 2 
Cn = 245) . ' Cn = 207) 


Year 3 
Cn = 103) 


. "'1 

Administra^on ^ or 
Supervision 

-J . •* 


27.8~^ . 3I.4L: 

• . ' 


29.1 


Teaching x - 


17.1 . 16.9 


19.4 



.Pupil Personnel 0.8 

^search ^ 10,6 

» 

Field Development 5.7 

- . • » " 

Ministry Regional 'Office 3^3 

Library ^ 6.9 

Private Consultant ^2.4 

IMdergradiiate Q.O 

'M.Ed. ^Student , 3,3 

Ed.D. Student - 4.9 

M.A. StiJident . 3.7 

Ph.D. Student' 9.8 

... ^ 

Other 3.7 



'21.7 



1.0 
13.0 
5.3 
1.9 
8.2 
1.4 
0.5 
0.5 
2.4 
3.4 
9. 7 
4.3^ 



>16.0 



1.0 
14.6 
2.. 9 
4.9 

i:.9 

1.9 
0.0 
3.9' 
'4.9 
1.0 
8.7/ 
5.8 



m.5 



ERIC 



13 



23 



The di^.trib,utibn of users by their roles in various 
..organizations is presented in Table 12. In public boards, 
users were most likely to be ; administrators of sup'e^rvisors " - 
(57%) though other staff also usfd ErsO- services to a signi- 
ficant d,egree w-ith teachers, researchers and librarians^ com- 
^ prising another 35% of board users. All ' ugers from the > 
separate boards fell within the admini^tr-ative ranks While * 
users in post-seconda'ry institutions were distributed among 
administrators (33%) , teachers (53%); and students C7%) . ' Most 
OPSE users were ^students (43%) though other- s.taff categories 
were also represented. Regional Office personnel were the 
largest (60%) user group at the Ministry of Education. 

It would appear that the patterns that were established 
in the first two years of the service continue" to hold. EISO 
is used as a resource. primarily~by administrative personnel 
in school boards, by faculty in-^^ost- secondary institutions, 
•by doctoral students at OISE,- and by administrative and 
Regional Office staff at the Ministry of . Education . . 

. Purpose of Searches 

Among all users, the mos^ frequently cited purpose for 
'doing a search was for term papers (Table 13)." Research and 
development reports, policy development, speeches or articles, 
personnel or recruitment and program development were also 
listed as the purposes for placing EISO searches. Users 
tended to. approach EISO as a rfesxjurce to supi^ort their primary 
professional role. For example, within school boards adminis-. 
tration was the most frequently mentioned role -for doing a 
.search (Table 14) . The^e administrative personnel performed - 
searches .to aid them with their responsibilities for program 
improvement (23%), personnel recruitment (17%) , policy develop- 
ment (13%) , and curriculum development (13-%) (Table IsV. 
Though it would appear that a few administrators did place 
searches, to help with their graduate studies, most, sought . 
information for purposes 'consistent with their ^professional 
positions. . ; . ..; 



O . ■■ ■■ ■ 14 

ERIC , - . -L-^ 



mEi2 . 

PROFESSIONAL ROLE BY ORGANIZATION' FOR YEAR 1, 2, AND 3 U» .(N =;245, 207, 103) 

I A ^ 



Organization 



* 

. Role . , . '. 


Public Board 


oeparate lioarci 


■Postsecondaiy 


■ OiSE' . 


. Ministry of; 
Education 


Other 






Year 




Year 




Year 


• 




■Year 




> 

: Year 






Jear 




1 


2 




1 2 3 


1 


2 


3' 


1 


2- 


3 


1' 2 


3 


1 


2 3 • 


Administration or 
Sipervision'^ 


r * ft 




57? 


641 mim 

, . 1 








' Li 




1ft 

it': 


\ 




ii 0 




Teaching or 'Pi5)il 
Personnel. ■ 


26 


6 


17 


,9, 18 0 ' 


25 


■49 


53 


14, 


14 


18 


0 0 


0.. 




15 9.,' 


'Research 


6 


11 


13 


18 0 0 


0 




J 


13 


il7 


15; 


9 ,14 


J' 


22 


: 23 36'' 


FieldjDevelopiiisnt or 
.Regional Office 


0 


4, 


3..' 


0' 0 0 


0 


3 


c 

0 




10 


w 


73 .' 29 


60 


6^ 


0 9^ 


Library 


10. 


17 


1 ■ 


5^00 


8 


,. 3 


0 


1 


.8, 


0 


•0 • . .7 


0 


0 


0 .' 0 


Master's Student 


0 


0 


'■0- 


5 0 0 


21 


5 


•7 


11 


8 


10. 


0 ' 0 


0 


6 . 


0; .0 • 


Doctoral Student 


l' 


0 


0 , 


0 0 0 


13' 


■3 


0 


36 


30 . 


nn 


0 0 ■ 


0. 


0 


8 9 


1 

Other 

1 


3 


4 


3 


, 0 0 .0' 


*7 


I 


0 


V 




8 


9 0 


0 


44 


31 37 


■ Total n = 


78 


53 ■ 


30 


22 11 2.' 


24.; 


■29 


15 

1 


90' 


77 


.40 

f 




5 


18 


.13 11 



• PURPOSE OF SEARCH ' 



'l 



Pinpose' 

^ ...... 



Other 



Percent 



Year 1 ' Year 2 Year 3' 

:.(n,=:386) (ii='-371) - . ■ . {n~- m) 



TemPaper .' ; , , ; ' , ' jLs ' ' '' 30.5 



Bibliogrq)hy '. . • 3.6 ' ' ;. 1.9 -':.' '[ ' L6 

Curriculii Development , 7.5 " . ■ \ .8.4' 5.4' 



.." Progra Improvement 12.7 ■ - , • , 12.4. . ■ 8.5' 

Speech or Article , 2.8 3.8, / • 9,3 

ReseM'andDevelopnient Report 2'8.8 '. , , ;, ■23.5 . ' 15,5 

Browsing ' . " ^ 2.1' • • ' ' 0.'8 . , ' D.8 "'^ 

Personnel or Recruitiiient . 2.1 . ' • 

Micy Development • ; ^ 6.7 .. « 10.5 " , ,■ ,* 11.6 



2.4 . ' ' ■ 3.5 ^ ' '^ 3.9' 



2? 



28 



■m 14 



;..RDif.FOR.MaLmio(m-i^ 



Organization 



# ■ 

; . Me for Which 
■ ■ RequesrMade - 


Public Board 


Separate Board 


Postsecondary 


OISE 


Ministry of 
■ , Education 


Other 


ft 

t 




Year "■' 




Year 




Year , 




Tear 




Year' , 


lear 




.2 


3 


2 


3; 


: 2 




2 




: l 


■3 


^ 2 3; 


1 

Administration or 
Supervision • 




! 51^ 




; 100% 




- 10% 


45 


!' or 


^"14! 




171 ■ 01 


. Teathing.orPi^il,, , 
^ Personnel . 


11 


5 . 


0 


0 


4 


5 1 


5 


■ ■ 2 . 


0 


\ 

0 




Research 


21 


23' 


17 


0^ 


29 


., 25 . 


' .28 


■.25' 


43 


22 ' 


25 '. 67 


Field Development or 
Regional^ff ice 


3 


* 

5 ' 


35 


0 . 


■4 


. 0 . 


*6 


■ 0 * 


29 


44 

< 


8 0.\ 


Library • 




0 


0 , 


.0 


0- 


6; 


2\ 


7 




t o 


-Master's Stdent 


S 


3 


0 


*0 


29 


40 


12' 


18 


0 


0 ■ 


' ,0 0 


Doctoral Student 




. 3 


,17 




11 


5 


34' 




0 


"0 


17 0 


■ ' Other ■ 




"■ '3 


8 


0 ■ 


13 


15 : 


7 


9 


7 


11 


• > 

33 v25' ■- 


■\ f • Totaln = 


62 


39 


, 12 


2 


45 


20 


86 


44-' 


14. 


9 . 


12 12 



eric; 



00 




, PKOKSSIOMII BY TO^ 



Role> 



1 

Purpose 


Admin, or 
' Super. 


Teaching or 
Pifil Pers. 


Research 


Field Dev. 
or Reg. Off. 


Librar)^, 


"Master's 
Student' • 


t 

Doctoral 
.Student' 


Other 


1" 


^Year 


Year 


Year' 


Year . 


Year 


Year 


' fear :,. 


Year ■ 


4 


/I 


2, ' 


1 


' 2 


■ 1 


' 2 


12 ,■ 


1 2 


, '1 


2 

h. 


1 


: .7 

M 


. l'- 2' 


Keeping Abreast 






' ,0l 141 






0M3l' 


01 01 


' 0^0 


' ■ 1 

01 


0^ 




01. 01 


Term Paper' 


14 ' 


17 


tL 


38 , 


A 

. 0' 


13 ■ 


•P" ;13 


6' 0 


100 


.80' 


84 


86 


>7 ' ;38 ■ 


Bibliograpliy^' 


n 


. u 


\ 

J 




A 

0, 


0 


0 0 ■ 


24 0 


. 0 


0 


. 0 


7 


0 ^ 0 


Curriculuni Dev. ' 




iO 


'4 


■ 4 


7 


A 

0 


27 0 


' 0 50 


0 


0' 




•0 ' 


31 .0 


Program Iii|)rovein5nt 


r 


2S 


.5 


10- 


11 




27 13 


24' 0 


A 

0 


A 

0 


4 


0 


23 . 0 


Speech, Article : 


7. 


3 


5' 


14 , 


0 


33 


0 13 


6 0,, 


0 


0 


0 


0': 


0 ,25 


Research § Dev. 


'19 


10 


32 


1^ 




27 


33 - 25 


12 vO 


0 


20' 


12 


7 


8 ^ '25 


Browsing , 


■■ 3 


< 

0 


0 


0 


0 


0 




'0 0 


0 


0; 


0 


0 • 


0 0 - 


r ersonnei Kecmim . . 


■ 8 


17 


5 


'0 ■ 


0 


_ 7 


, 7 25 


6 0 


0 


0 


' 0 


0 


15 10 


Policy Developmenf. 


17 


13 


■5 


10 


15 


13' 


..0 0 


18 -SO 


• 0 


0 


0 


0 


15 13 


Other 

• ■ 


J 

4 


0 


5 


0, 


4 


0" 


6 0: 


'0. 0. 


0 


0 


0 


0 ,, 


0 0 


^Totaln^= • 


65 


30 

0 ' . 


37 . 21 


27 


15. 


15 J 


17 2 


8 


A 


25.. 14 ,. 


8, 8 
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Thrs quantitative data gives a general overview of the 
user population and their reasons for requesting searches.. 
To provide a more rounded picture of the "human motivation that 
prompts specific searches and the factors that contribute 

_._....t.owaiid_tihe_succes.sful-use of - the; information- we- muslr turn -toy - 
descriptive or qualitative data. Such dat^ is of course dif- 
ficult, to document and assesl. Often influences beyond.^ the 

- ^control of the. Individual are. overriding factors in whether 
or not information Is used. for the purposes for which it was 
originally sought. Another limiting factor is that evidence - 

■ o"f usage has been compiled thrpugh interviews that relied oh - 
self-reporting. Though" on'site. observation may . have provided 

• more reliable evidence, , these data are nevertheless presented 
•for the. insight they provide into some of the elements that 
determine use or non-use of search results . 

'-l ." ^ Case Studies . — 
The' Psychologist . 

■ ■ . - ■ ' , .J ' ■ ' 

Dr. Barton is the dynamic administrator of Jpupil person- 

^ nel services for a separate school board in the Metropolitan 

: Toronto area. In her 'late thirties she holds a .doctorate in 

her specialty, child psychology, and has had five years 

experience in her current position. She is an active member 

of her professional organization, and has written severail - 

papers that were pither 'published or' presented at conferences. 

-By her own admission, she is an, avid information seeker. 

• placing a high value on keeping up to. date in her .field. She 
.. l>.eard about the existence .of colleague's and decided 
^ to ..place a sear.G she, foiind' herself- chairihg a group of 

parents -and teachers, who fel.t° there was an increasing need for 
a play therapy program for their children. . "But one of the 
problems we had, sVe explained, "was that we did not have 
anyone here who had had extensive training or experience, in 
play tl;ierapy.- So one of the purposes of the group was to 
educate ^Jseif as.well as to ecKicate other staff members who 
.would .b^. lis iri-g . the-:. ser.vicg., ".. . ' ,. ,. 
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■ A telephone request was submitted to the search analyst* 
who was able to^ retrieve sixty-three citations on play therapy 
from the Psy chologioal Abstracts data base. The bibliography 
was. mailed to Dr. Bart9n who, subsequently ordered twenty- four 

^_j ouniaJ^„article-s^_. o£--which„- ElSO- _supplied~ ^her- wi th- .copies -of 

twelve. These were divided up among the members of th,e com- 
mittee who used them as a basiB for seminar presentations to 
their group. The information thus presented broadened the 
gr.oupVs view of play therapy, according to Dr. Barton. While 
they had previously 'been using a traditional .kind of . definition 
they now encoujixered an unstructured, non-directive therapy; 
program. ^^^x^^S^y ^ were also introduced to o'ther ideas for use of 
the play room, such as having mother and child go in together 
for observation. T'oys we^e p.laced in waiting rooms and social 
workers' offices; the play room began to be used for assessment 
purposes; programs were expanded ,to educate the mothers as IfTell 
as. the ^children. Play activity became a means of teaching 
social skills . - . . 

Although the psychologist has becoDje a repeat user of . 
the .service} she expressed only moderate satisfaction' with the 
bibliography and materials she received^ as a- pe'suliy/<ff : her 
automated search. / Her enthusiasm, was dampened when sh-e dis- 
covered that the ^ata base ^searched - for her did not contain 
early classic worlds written on play therapy. ' There , was also 
some, question about whet^her she had obtained all relevant 
items in the data base since the texm ^tplay therapy*' seemed 
to be a recent one. La^t, she had had some^ difficulty " 
obtaining all the relevant art i^cl'es that had been .ei'ted in 
the bibliography. - : ^ ■ .; 

The Junior High .Principal " - "' /' : > 

Mr.. Albert had been the jtinior high . principal for thrise 
years. He was not an* especially avid information seeker but 
did make efforts to keep abreast bf.liis fieid. . Iii the evenin^gs 
he was purs^uing an M.Ed, degree at OISE.' 

At the time of his. search, he was spear-heading a. drive 
in his ,school to revise the music ^^gram. f rom a stri^ctly ^ 
instrumental- to -a", m^ diversified ;-one, ^ Having, learned, .ab^ut' - 



EISO through his graduate studies and from. brochures mailed 
to his school, he decided to telephone the searcJi analyst for 
help* As he put it: . 

' ^ We wanted to se'e>what else '^liTad ^ been doiie in 

^. other^J ur i.sdictapns^ ^ what^ else^^ h j_ 

written and' tT^:^^V acnd was available in -the 
.c field ratJier ^^^^^s^tart* f roin^ We • 

• figured a. 16t^^^P^0Le, •h.ad^ probably done v 
things'.bpfore^^^^^^^auted ^J:^ avoid making 

the saitie 'jriis^t^S^^^l^^^f^^ ^ , 

7 -. ^ --■ ' . - . '-^ - -A---. - 

Searchiag\t*ie ERiG 'da^a^^Sas^ tTae^ search analyst 
retrieved eighty-three citajt j.on5^»*that . were^ printed of fline^ 
and. mailed to the ' principal'. ,. He theii turned over the biblio- . 
graphy to His two music te'achers who .reviewed it and went to 
their own board's professd.onal. library to examine the relevant 
articles and microf-iche..". They rated • their satisfaction with 
the bibliography and materials .between "medium" and "high." 

'Basically, they felt, that the ^search-'had been valuable but 
that too much ^material had been found. They were frustrated 
at not having the time to rea^ twenty*- fi.v,^ .microfiche and 
thought that- in future they would confine themselves to 
defining their problem more narrowly. They ■ also felt -that in 
future they would visit the search analyst in person rather 
thaii communicate via telephone. TOiis more personal method 
woizld, they felt, better enable them to clarify -their "muddle" 

'as; they, put it, and. to -understand the search pro'cfess better. 

Th^e'se difficulties aside, the mysic teachers, with the 

principai'-s support", initiated a diversified music program 

for the. firs,t half of their grade seven unit . The prinpi'pal 

also saw other .positive results from the search. : ^ ;r 

; - ;. There was the indirect benefit ; of • expos ing ■ 'j 

the music istaff to~* this kind of information. - , 
. 4^.... '. From* now'.oii,;- wheri^ they , encount-er-. ;a problem,-. ' ' 
'.they are more l}ilce'ly to '^sk whether anything 
has been written on it 'in the literature 
sinpe they haVe at least on one occasion 
asked the kind/of question that would not 
have been asked before. 

The principal testif&d to the success of the new music program 

and credited many of the successful components to ideas gleaned 

from materials obtained through the EISO search. . 



The Program Coordinator * 

At the separate school board in Hamilton the program 
coordinator was sitting on a C9mmittee lojrking into the. role 
of the elementary school principal • The '^group was especially 
concerned with what their own system could to to "prepare staff 
f^X "the principalship and how principals were evaluated. The 
committee was composed of an area superintendent who acted as 
chairman, three principals, a teacher, a vice-principal, and 
the coordinator. They had been working together for a year 
and had focused in on a Management by Objectives (MBO) evalua- 
tion model. Having settled on a direction for themselves, 
they decided to consult the professional literature to find 
out what was going on elsewhere. ^ 

The* coordinator filled out tne searcti requesjt form on 
a brochure and mailed it in to the EISO office in Toronto, 
The search analyst clarified the request over the telephone 
and eventually retrieved 112 citations on training for and 
evaluating the role of the principal* This extensive biblio- 
graphy was divided up among the members of the committee 
each of whom gave ',a talk about the relevant items they had ^ 
examined at their local learning resources center. They were 
highly satisfied with the bibliography and the materials.' The 
coordinator expressed their feelings this way: "Personally, 
we were delighted w.ith the whole thing. It gave us what we ^ 

wanted very 'quickly . .V/hat^ resulted from their search? She 

■ ■ ■ . . ■' , > ' 

continued: 

: • - ^ . Well, out of it came the document ^'The Rol^ 
/ of the Elementary 'School Principal*' which:, 
is -ptou Iri use". . We have a -whole evaluation 
process . f orVprincip.als in place based on a 
Management^ by Objectives model. "We had in 
as speaker one of the - authors of the , ^ 

articles . ' 

In terms of ^the preparation of principals , the search brought. 

,to light the^ notion of interns:hip programs and a sub- committee 

was "set up to examine the feasibility of that specific idea. 

Summing up the contribution the EISO search had made to the 

committee's work, the coordinator said: - 



It gave us the kind of information that^ 
in spite of the fact that we already had 
the direction, lent support for our posi- 
tion; It couldn't be regarded as just some 
silly thing we thought up all by ourselves • 
It was comforting to be able to wave the 
literature around when presentations were 



The use of the materials did not come to an end with the 
original committee, however. As indicated, a -subcommittee 
was' formed to investigate further the internship notion. They 
too used the articles retrieved. One .of the committee ' members 
also passed on the materials to a colleague responsible for 
• staff development, at a .local hospital.- 

■■ ■ ' . ■ ■ ■ / 

The . Superintendent , . • - 

r George Baxter is a well-known opinion leader in local 
and provincial educational circles* He. has held increasingly 
responsible positions In school boards' and at the time of his 
search )was serving as superintendent of instruction with an 
expanding, innovative school ^board adjacent to Metro Toronto. 
His search was prompted by a series o£ leadership workshops 
that he was running for his sta^ff. 'He wanted specifically 
to find out more abolit the phenomenon of "group think," 

His search was* performed on the* ERIC, Fsyohologidal 
Abstracts y and So^ciologioal Abstracts- data bases and he 
claimed to be very satisfied with the results. He*, abstracted 
the articles he .examined , created overhead ^ transparencies 
showing th-e salient points p£ each item, and with, his packages* 
on group think gave five workshops to a group of department 
heads. Another group of teachers learned about these sessions^ 
and requested that similar presentations be made to them. 
They in turn borrowed the superintendent's materials and used: * 
them. with some students in their- secondary schools . The same ' 
material was further used by a group of principals and adminis- 
trative staff within the board studying the decision-making 
... . .. .... 

process. . ■ ^ 

Using the service has become the accepted first step of 
most vorking committees in -this board. The superin^ndent 
commented on this widespread usage: **It's the best money 
we've ever spent." As to his- own use of EISO, he stated: - 

ERIC • , 



IT I want to give a talk\or I'm developing 
a paper, very, very often I won't do it 
• until I do an EISO' -search\ It's good to 
know that I can get resources if I want 
them. It's nice to k-now the service exists 
whethier I want to use it or not, and I do 
J__ _ _ us_e it. I^t has that^ kind of comfQrting 

b log re air e F f e"c t " ; ~ ' ; ~ 

The Ph.D. Student - . 

As a doctoral student in curriculum, Mr. Abott was 
interested in doing a research ,4>aper on reading comprehension. 
Though he spent forty- five minuites with the search analyst . 
and they went through, the ERIC Thesaurus together, the student 
was unable to define "his topic in much detail. There. *\fas some- 
doubt about whether he was .concerned with reading comprehension 
listening ability, or the- e'ffect of the teacher on student* . . 
achievemenit in either, neither, or both areas. ^ 

An e-xamination of the S,earch Request Form completed by 
the search analyst shows that two searches were performed., _ 
one on the effect that a teachej^r's attitudes ai^d teaching 
methods have on student listening comprehension and another 
on the effect that a teacher ' s attitudes .and teaching methods 
have on reading comprehension. , A total "of thirty-four cita- 
tions were retrieved and sent, to Mr. ~ Abott. . - 

He expressed di.sappoinrment in the bibliograpky that , •■ 
resulted from his search. Too much of the material, he felt, 
dealt with the comprehension of other languages and with the 
teaching of reading as a subject whereas his. interest now lay 
."in the more" theoretical stuff, how' the brain -operates , how , 
language is developed, how people understand and comprehend' - 
language . " 

The search results were €,il'ed in his office where they . 
remained unuse'd. At the time of the . follow-up evaluation 
interview, a year later, the student was still struggling, to 
define his research topic- ; - - 

•The M.Ed. Student ■ . ^ V ■ - - • 

As "a teacher' of industrial arts at an intermediate. '. 
school in a Toronto boxough , . Mr . Taylor had students from both, 



middle^ and low^r-ijfi come areas in his classes. He. had -begun 
to -jFeel that- students ffoiri thfese two different types of back- 
grounds .regarded industrial arts as a subject quite differently-. 
So as an-M^Ed. ^studen^t at OISE, h-e decided to focus the topic 

As he put it: ^ 

r.was attempting ^to find out whether in fact . 
\ cKildren with lower socio-economic back- 

grounds would have .. different attitudes to / 
* .industrial, arts than those from high SES 
• ' backgrounds. I was using, the search tb find 

out what the "Idteratu said. . - . - 

The search-^analyst devised a search that, retrieved' • ^ 

•thirty- seven citatipn's . . These proved ' to be of li-t:tle- use to 

Mr.^ Taylor /however . He glanced through the bibliography 

rapidly, and, pressed- to hand in his as^igninent , deci^ied to . 

rely on: articles t'^at had been hajided out in class. . He di^d - 

.not order any original materials, or otherwise use the results 

.o£ the searcK in any way. - He ''reiiiiembered having difficulty. \ 

with the" format of the ' bibliography and fee-ling, that' the ' ^' ^ 

effort requiite^d J>f him to order materials, read; and digest 

them' was'', too. .great a burden when added to his full-time 

^ teaching/load. Por his ' course assignment , he submitted a 

term .j)aper;.,that-- dealt: jcith experiences in his- pim* '■classroom. 



The Primary Teacher . . - ' . V 

At a public school board at . theVopposite end of Metro 
Toronto, another industria-1 arts teacher was ^sp^ placing an 
EISO search.' He was Tiopihg to apply, for a leave of absence 
and placed liis search to bolster, the presentation .he needed 
ito present to his board to be granted .leave. 

-An examination of his Search 'Re.q^uest Form' showed 'that 
the; search analyst combined' the concepts, of silow learners, ; 
perceptual -motor coordinaJzion,^ and "classroom materials to 
yeild a bibliography of fifty citations .'^^ ^ These- proved, to - 
Abe ' irrelevant to the industrial -aYts..;teacher, who placed his 
search. output ii/ his ^ -file and fb:f got 'about it. When asked if 
te c'auld explain what had gone wi^ong, he' gave . the following 
.answer: ; ■ ■ ~ .y r^"' ■ . • 
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The project. for which I was applying for funds 
dealt with the work-^bench approach to teaching 
. industrial arts in the primary, grades. I 
/•didn' t really know what to ask for. I knew - 
what the' general topic was i what the* general 
•idea was, but I didn', t really know any of the 

^ '_spe|j;if4_c things lithe .search .analyst wa^s 

asking me- because I hadn*t studied it before. 
The girl' on the phone wanted to know more 
ajbout it and I couldn't tell her more* I 
. ' ' couldn^t explain to her what I needed because 
I didn't know. All. I knew was the work-bench 
approach* ' . 

• ■ ' - ■ ' ' * 1 " . ... ' " 

Factors -Contributing to Use or ^ ^ . ,^ 
Non-Use of ^Search Results . * - . 

The seven mini-cases described above, though they 
represent an admittedly limited sample of users, nevertheless, 
point to some ^f actors that -either - encourage or pose barriers 
to the^use information received as ^ a result of placing an 
EISO search. These factors may be evidentr during bjoth 'the 
:pre-search and post-search , and are described in Table 16.- - - 
Circumstances pertaining to the* intervening search negotiation 
are examined elsewhere in this reportl . 

Of the seven users shown in Table 16, the psycholGgist ,• 
j'unio^' high principal, program coordinator, and superintendent 
all illustrate degrees of information use while the two. ^ 
students and the primary teacHer are examples of non-users, 
to what .factors-can we attribute these differences? 

' The psychologist, it will be ^recalled, was an active 
Vesearchep with, the typical profile of^an innovator. ..She 
was also the .chairman of a comjnittee that was- motivated^ to 
find answers to a prjsssing problem and she had the authpx'ity 
both to >5:€e" t/hat a search was placed and t4iat its-^results * ^ 
were implemented-?. Upon receipt of the articles, she. distri- 
b^ited ' th-em"" to her committee members who then assumed j oint* 
xesponsibility fdr^ putting' new programs into efjfect.. -The 
negative aspects of the search, accQrding to the psychologist., 
.related tG"^he\ fact tlaat few '"cla.ssics.V in the fie'ld were * 
retrie,ved,^ that the concept, "o'f- "play therapy*' was a -relatively 
new- te'rm and therefore might "restrict the numb.er of relevant 
citations, retrieved, and the! difficulty ^of obtaining someV 
original articles . ' . ^ ' . ^ 



. TABLE 16 . 
FACTORS MJING-INFDMnON USAGE. 



User 



Search Phase' 



Pre-Search Phase. " 



Psychologist ^ 



Pros 



Cons 



1. Active reseaf^her ,1. No "classics" 
2: Chaiiman o^feconmit- . 2. Newness of "p% 

tee >dth Wm- = ther^" 

tion need.'H^^:' 
3. AutiioVity ^6' li^le- . ' " 



Post-Search Phase 



Pros 



Cons 



1. Articles to canittee 1. 
sobers 

2. Comnittee responsible . 
for inplemaitation 



Difficult/ in.ol 
articles 



Junior High School 
Principal 



•1. Postgraduate student ' i; Telephoned request 

2. Dissatisfaction with 
existing programs 

3. Authority of .posi-- y . 



1. Support of principal 1. 

2. Literature reviewed , 2. 
by teachers 3. 



Too niuch material 
Too little tiffle . • 
Inadequacy of telq)hoiie 
interview- 



Program Coordinator 



1. .Qmttee-initi'ated 
search- 

2. Well-defined problem 

3. Authority-,. 



1. ' Articles to conmittee 

inenbers 

2, Each juenber gave talk- 
^.'Author invited as' 

guest speaker 

4. Infoiiation siqjported 
position . .■ , ■ ' . 

5. New infonnation pro- ' 
vided 

6. ^)eed 



Stperintendeat 



1. Oj^oi leader " 

2. Infbrnation needed 
for workshops .', 



1. Infonnation ntade into 

packages ■ 
2 . Warkshops given to 

departnent heads , 



Ph.D. Student 



1. Postgraduate 
student .,- 



1. Undefined infonnatioa 
, problem 

2. Inability to cotmuni 
cate 



1. Irrelevant material 

2. Different, topic 



•M;Ed. Student 



1. Postgraduate, 
studaat 



1. No infonnation need 

2. Limited course 
requirements 

3. Job ^sures 



1. Format of bibliogr^Iy 

2. Quantity of infoimatioa 

3. Lade of urgency 

4. Efiort required to 
obtain docuaents 



Primaiy Teacher 
ERIC~^ 



t/lhable to ejqjlain 
topic 



1. Irrelevant material 



- The junior high school principal, though not as involved 
in research as the psychologist, nevertheless also made efforts 
to.keep -threats of his field and was pursuing, an M.Ed, dd'gr^e- 
in his. spare 'time . While he did not head • a committee , he wa$^. 
the leader of his school with prime responsibility for its.'. . ^ 
progifams.' 'in this role, he recogiii zed "the inaxiequacies<^of hi? 
music program and determined to correct them. To achieve his 
aim, -he gained th« support of his two music specialists, ^ 
encouraged them to review the literature he had obtained' and ' 
to -initiate appropriate program changes.: Though the informa- 
tion obtained was used,' the results of the search presented j 
some problems: too much material was obtained; there wa? not 
eno'ugh to read it all; , there was some dissatisfaction ;, 
with placing the request by telephone rather than in person. 

. The program coordinator, though not chairman of his com- 
mittee., was", also a m,ember of a high-powered groujp that had been 
working together for a considerable p.e-riod of time looking at 
the role of the elementary school principal. -They had deter- 
mined what their information need was and charged the program 
coordinator with the responsibility of placing , a^ search . Sub- 
sequently, -each member of the committee made, a presentation 
using the documents obtained as a , result of their search. The. 
information they received was relevant to their concern, sup- 
porting their ideas and -presenting them with new ones., - ^ 
The superintendent was" not a member pf a committee but. . 
he was a charismatic leader within his own school "board as ■^^>. 
well as in broader educational, circles.. „H^ . exercised one^ v . 
aspect of his prof ess iona.i role by presenting wo'rkshop.s for. . 
his staff . It was to develop these presentations, thatv he- . 
placed a .-search . The inf ormat'ion he received Was transformed. = 
into packages that- we're, used at his .Workshop-s and distributed 
•to other teachers requesting them. The, superintendent was 
known to his staff as an -avid information seeker and user . 

" To 'this point, each of the four requestors represents ^ 
an active information user. The factors that enable^ them to 
become users,' or facilitators, of information use , are remarkably 
similar in each case. All had personal characteristJN^s .that 
are normally associated with innovators .. They were active. 



.involved, highly educated,^ outgoing professionals • But most / 
important for this ^study, there were'" aiso^ articulate, able to 
define, and Vefbaiize their information problem,^ aware of the 
sources they could cali upon, and-'possessed with a healthy. / 
belief in the efficacy of research- and a curiosity to. learn 
about what was going o.fi elsewhere*^ Furthermore^ they were. 

also highly motivated. This motivation may have resulted, from 

. . • ^ . . ^ - 

personal or professional needs' but . in ev-ery case there was a" 
^ real ,^ concrete-, identified problem to be sorlved or si;tuation 
to be, changed • Thirds each user was. in a position of "^authority 
to initiate a search himself or to cause otliers to do^<so. . This ; 
.^uthority may have resulted from legal piosi Cth^principarO 
.Expertise .(the psychologist) , /delegation, .(the progtam^-Soordina- . 
tor) , or charisma (the superintendent J but ; in each case it ^- 
. ensured that a search would be made and ^i ts results considered, 
and used, if at all ' possible, ^ A 

' While all of -these -variables wehre crucial in the pre -sear ch^ 
phase and continued to be importarht later/, s till other element^ 
became important during the post- search period* ' Here^ . too, we ' 
note that all . four cases h^cve certa.in-<:ommon' features • "\Fbr 
V example, in each case the" original documents obtained were • 
passed on to^tho^e responsible for effecting change • T^e in- 
formation Was shared and in^some cases adopted by committee or 
staff* members • Join't ownership and ^tpmmitmeht 'Was\ developed* . . ^ ' 
^Thos.er who received, digested^ and adap^ted the in^orma:tion -tb ' 
^ their own needs were.' alsb those with ^responsibility ''■fox' ' :-''^-;:,^ 
Qarrying through' tfte'-resUlts of*^^^ plans, > i<e'f7Nj^he seek^ 

the receivers , and the users were all the same peop!D^. In . the^ . 
case . of the superintendent , ' there was also heavy SQcond.ary , use 
of the information packages. .Most of this, type of usage can be 
attributed to^ the charismatic authority of the superintendent ^ 
'to the raised awareness of his staff, and the"* value attributed 
to inform^ation use by him*' , 

. The last three cases on Table 16 illustrate those factors 
that p^se barriers to information use. The doctoral student" 
was still in an early stage of ]^is program and had .not ' yet. ' 
defined his research topic with 'su£fxcien had 
a -vague ' notion that ' he might be interested in the effect" of 
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teacher attitudes; on student achievement but was really unable 
to be more specific. Iji^ny event, shortly after placing his 
search his interests shifted to another a,rea and he found the 
bibliography he .received of limited value.- 

.-The masters' student, while he had a reasonably well- 
defined topic, really had no need to place a search. His 
instructor ha*d handed out copies of relevant articrles to ' his 
class anc^^^he course assignment ^did not seem to require more ., 
extensive resources. There is therefore some question as to • 
why he placed the'-search at all. When 'he received his T^iblio- 

grap^hy^^he was discouraged ,f rem obtaining or using original . j 

--■ ■ .*. 

documents by the time: constraints placed on him by his regular 

job , the . discouragingly large amount of material available, 

the^^ormat of the bibliography^ and the fact that, he' a.lready^ 

possessed sufficient resources to complete his course assign^ 

ment* ' ; * ; ' - 

• . ■ ■ • ■ ■ ^ ■ 1. ' ■ , ■ ^ . ^. ■ 

The'' primary teacher, though havin'g a reason for placing 

a search, wais hevertheless. unable to explain to the search 

analyst .wha|:, his topic was about . He was limited to a single: 

approach in teaching industrial /arts and was unable to. expand 

'upon' it or ^place. it in a context for. the^urther 'edificati.on *. 

of the search\^alyst. ' By his; own admission, he knew very ' 

little about" his top i43^ and. did not know what ^to ask- for. Not 

'surprisingly, .he found :the bibliography he received irrelevant 

- -Trom. . the^e brief sketches , we' can- sSe. tha't the . non- users , 

bfv information also "share some common characteristics . Among ^ 

■ .:; ■ . •>'• - ' ■ 4i ■ ■ ... . • . - ' - . - V . . • . - 

these are ^the lack of a clearly def ined ;^pxpblem and/or the . • . 
inalDility to articulate that.'problem to the search -analyst. 
These were especially true in the cases of "the doctoral : 7 
student" and the pfimary. teacher . - Lack of motriv^ion or> real 
need was a deterrent to usage in the c'as*e^of the^ masters' - 
student although he was also discouraged by the format and 
quantity of material, the effort needed to acquire documents , 
and the time pressures of his job and course ' requirejments . 
These .latter factors" Were operative most, clearly in the post-* 
.'search phase . Lack of commitment in this later stage also . 
prevented the, doctoral student and the primary 'teacher from " 
following through with their original -concerns. *o 



; ' Conclusion , ^ 

It is interesting to note that whereas .in the cases o£ 
sUbcessful information use each>«2er was spurred oji by the 
need"' to- report to or work with .^ta'^ff or ^ 

users; were all. loners. -It might: be- expetted," therefore, that 
-they would have to display greater independence, self-reliance 
. - and personal md.tivatipn than the successful users and would 
not have the rewards of immediate positive feedback In the 
cases dealt with here, th'ls increased burden ^o-^ responsibility 
without positive- external support" created a barrier to use 
that was not overcome. 



I 



■ • :. ;CHAPTER '3 . ' / 

; ■ ■ ; ., , . -user satisfaction with eiso ^ • •- 

; One of t^^ most difficult aspects, of an 

online search serTiice t.o, evaluate the satisfaction of the' ■ 
user. User satisfaction is /significant 'because it will affect 
the ultimate success of the service: a satisfied user is more' 
lilcely to become a Repeat user and an advocate of the system 
that has met his nee^. And it is the burid-up, of a steady 
clientele and the spread of its success , by word of mouth' that 
will help to ensure itsv long- term operatq^on. Measuring user 
satisfaction, liowever, is .fraught with difficulties for we 
are trying to assess ^ what is , after all , a state of minUi Not 
only: is it highly subjective, but it is in all likelihood ^ 
.subject to frequent shifts and changes. What may be judged , 
satisfactory at^ one point in time may be considered less so 
at another . So the problem of when to assess satisfaction is 
a -real one. • Equally troublesome is What to. measure. A -search 
service consists . of numerous, .elements that- may each ■contr;Lbut'e ^ 
to the ' peroeived s at is 

tion of : the user . Such elements may 
include everything- from tie user's interview with the search: 
analyst to the , speed with which original documents are received. 
Performance- at e^li -of . numerous stages will influence' the satis- 
faction, of' the user., • 

: Recognizing this complexity the Educational Information^^ 
•System for Ontario developed" an evaluation scheme based on a ' 
systems mo4el' using psychological , sociological, and economic 
variables within an -adoption of innovations framework. Thi s- 
schema aiid- its reliability were described in detail in earlier 
i-eports CAuster and Lawton,- 19 76,. pp. •56-84;- and Lawton and 
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Auster^ 1977, pp. 30-68) It is the purpose of this chapter, 
therefore, , to present satisfaction- data for a three-year 
period and to -examine an aspect of the service not reported ^ 
on 'before., namely comparison of the satisfaction levels of 
users who did and did not go through Educational , Information ' 
.Consultants (EICs) in order to use the service. ' 
^ - The five scales used to measure user satisfaction are ^ 
; Ixsted in Table. ,17. These are.: Publicity Materials and 
Directions , Quality of Service , timeliness of Service^ 
Quality of Technology, and Value o.f Bibliography and Matefials 
A sixth composite scale measures Overall Satisfaction. ^ " 

Yearly Satisfaction Averages 

* '* <■ ■ ' .. ■■ . . ... 

'a comparison of the average satisfaction of.EI^O users 
over the three years of the servicers operation shows a ^ 
remarkable consistency. Changes in average level, of satis- 
faction ranged from an increase of .20 for Timeliness of^ 
Service to a decrease of ^04 for Quality of Technology. There 
„ was virtually no sigiLif icant variation in perceived satisfac- • 
tion^among EISO user stover the three -year period with .the 
average user being highly satisyfied with the service he 
received.- This high average satisfaction wa.s expressed by 
• means pf 2.61 and 2. 78 for .Overall Satisfaction and Quality of 
Service, respectively, on a; three-poirLt sCale.^ Table 18 shows 
that fewer than 5% of user's were- not satisfied with , the out- 
puts of the service, 'the Value of the. Bibli<*graphy and 
Materials, during^ the most recent year of operation. This^ 
represents a. significant improvement from the^^^m 
who expressed reservations the previous year or the almost 
10% in. the first year* Clearly EISO continues to improve , 
-with over 95% of users being satisfied 'with the service pro- 
vided. ^ . . . " . \ - : . : : ■ • 
; A more detailed breakdown of the five rating scales in 
Table 19 reveals similar responses to 1:he scale items. On . " 
the same three-point scale mentioned earlier the least satis- *, 
factory item was ^judged to be the time taken to deliver . - 
materials from EDRS with a mean' of 2.09 and no aspect of the 
service was judged, to be unsatisf attotV . V It should be noted - 
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SATISFACTION SCALE MEANS, -SIANDARD DEVIATIONS,, AND FREQUENCIES^' 



Scale ., 



Year! 



Satisfaction 



Year. 2 



X s ■ ■ ii 



' Year 3 



n' 



Piilicity Materials 
and Directions 



2.57 LOO 



2.58 ■ 0.45 ..198 



2,56 0.40 ■: 114 



Quality of Service 



2.31 • 0.60 200, 



2.76 -0.39 -211 



2.78 ■ 0.37- v:- 110 



Timeliness of Service 



.2.31 'VM 25 2.40 ,0.44 . 36 



2.60 0.42 19 



Quality of Teclmology' , 



Value of Bibliography 
. and Materials 



2.42 , 0.76. ;-56' 



2.53 0.38/ 87 



2.24 : 0.87- 133 ■ - LSO ■ 0.67' 179 



2.49 ■. ; 0.40 47, 

■ * 

2.36 0.57 102, 



Overall Satisfaction 



2.61 0.77 180 



2.60 .0.35' • 188.. .• 2.61 0,;-30 , 100 



|Scale mans relate to a three-point scale: 3 = high satisfaction, 2 = msdiuiii.satisfacti<in, and 1 =.■ low satis- 
■.faction.' . , ■ . 
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•TABIi 19 



SATISFACTION IH MEANS, SliWD -DmiONS, AND FRHJDENCIES 




Micitj ht^naU 'and Divectim 



. ' ■ Accuracy and.conjrehensiveness 
. of publicity materials. 

w Adequacy of directions for . 
9 ^ , . submitting search requests^ ( 

^Convemce of arfangeinents and ' : 
• adequacy. of direai(His^ ; . 

•Adequacy of' directions for ordemg 
copies of materials . ' 

. ■ • ' ' ■ J ' '■• 
. . . ■• ■ ■ : 

. ' ^ Convenience ,of arrangements to 
" . ^btgin EISO searches^ ... f. 

Convenience ^ arrangements and ; 
" ,• ■ . adequacy of. directions^' , 



I": / 



■2,45 0.66 -112, 
.2.68 0.55 ' 143 



2.51 . 0.69. .182 



2.-84 . 0.41, 237 



i-MpfulnesSvof Search;Myst or EIC< ■ .2.88 0.37 . 225 
■ TiiDe devoted to -search .interview^ ; ■ 2.71' 0.58 208 



2.47 0;61''208 ,./2.42 0.55.^16 



2>70' i52:. 225; 

■ ■■■■■■■ r 

■ , ... ■. >... 

2.55 •0.62' 212 



:'2.70 ■ 0.52 225 
'2.8r 0.47''' 2li 



Zr.' 0.51' 124 
2.57 0!55 122 



2.69 0.51\ 124 



2.86 , 0,37 



'me 19 (Cont'd.) ' 



Satisfaction . 



; ■ Scale -Item ■ \ 




Year 1 






. leaT Z 






■ Year 3 




■ s 


x' 


s 


n 

■ ■ ) • 


' ' i' 


c 




A. 


. 5 




fifneliness ofSeroice / ' ■■\ ' ' 




















Tune taken to deliver bibliography 


■2.64 


0.63 


224 


: 2.50 


0.65 


227 


2.66 


0.51 


125' 


fromEDRS . . 

r 


2.24 


0.83 


33 ■ 


2.06 


0.66 


54 


.2.09 


0.71 


34 


' . Time talcen to delivef.nHterials ' 
• frokEISO . . • 


2.42 


0.72 


53' 


2.54 


0.61 


. 65 


/ 2.86' 

'I 


■ 0.44 ■ 


.29 




















1 


Length of bibliography • , 


2.41' 


0.66 


227 


' 2 51 


0 60 

Iff Uv 


916 






110 

iio 


Readability of microfiche copies ' : 


2.35 


0.70 


•93 


.2.38 


0.59- 


123 

# 


2.35 


0.63 


60 . 


Availability of microfiche' readers ' 


2.58 


0,67 


121 


2.56 


0.63 


;il8. 


2.55 ■ 


'0,62 


62 


, Mm of Bihliograp}^ md Matends ' 




<• 
















Bibliography itse],^^ 


^ 2.19 - 


0.69,' 


^221 


,2.33 


0.70 


216 


. 2.40 


'0.63 


116 


• Materials located.vi^a bibliography ■ 


2..25 


0.68 


134 


: 2.28 


0.72 ■ 


■ 182 ■ 


■ 2:31 


,o.6i:. 


lo: 

1 



^Separate questions concerning convenience of arrangements and: adequacy of directions for ?imittM search- ' ' • 
requests were asked inJear 1, but .the two ites were combined inlear 2 and Year 3.. • ■ 

2ltem on lme devDted to search interview did not ^ear- on Year 2 or Year 3 questicraiaires. 



t|iat .corrective action was taken to provide more effective 
document delivery by cOntr-acting with MicroMedia, a local 
firm, to provide paper copies from fiche. Thus the long wait 
formerly necessary for users dealing with ERIC Document Repro- 
duction Service (EDRS) in the U.S. has been eliminated. > 
• e Those aspects of the service that were felt^to be most 

satisfactory, with 2. 86 -on a possible scale of 3, were the . 
.time taken to deliver^materials from EISO, and the hjelpful- - 
ness of search analyst or EIC. Since the speed of the service 
and the interaction with the search analyst are two of the 
main distinguishing characteristics- Cand" theretfbre selling 
points) of onldne retrieval, it is gratifying to know that - 
these important aspects are being carried out well. On* th6 
whole,, then, these statistics, show a seicvice tha't has estab- 
lished high standards and is succeeding -in maintaining them. 

Monthly , Satisfaction Averages V: 

While it is -true that oh the whole, the EISO service 
is maintaining high standards, fluctuations do occur from 
month to month depending on the time of year, major changes ' 
in.system, operations, and the size of the sample for whom 
response's .were received. It is as well to recognize that \ 
these Variations do' occur- and in fact appear ;t:o be a normal .• 
part of the seyvrice ^environment so that undue anxiety or .* 
uncalled, for -shi'fts in policy are avoided; • . . • ' - 

- • * Among the most important measures asses'sing^the satis - 
-faction 6f users - with EISO ^re those^ evaluatxrig their jiercep-'- ' 
ti on about ;^e Quality of Service and. the Value of Biblio- . 
graphy and Materials Because of the importance of these „ .. 
aspects, mean satisfaction levels of users regarding* them> 
were plotted on a monthly basis . from July 1975 "through October 
.1977. An average Overall Satisfaction level. was also obtafhed. 
A graphic description of the results is seen dn Figure 1. • 
Trends during the tenure of the f irsf" search analyst 
were.-ranalrz.ed ..in E ISO's Second Int&T-Lm Report z/Mar'oh T9 76 - 
Fehrudi-y 1977 CLawton . and Aus ter , 1977, pp. 44-50). Briefly 
it was fpund that Overall Sjatisf action was remarkably steady, 
ranging between .2.4 and 2.7- on the three -point scale employed 



■ 'figure 1 ;■ ■ 
. average user satisfaction bym ■ ' ' 




The monthly average remained within ±0.1 units o£ 2.6. 
A noticeable drop did occur , however , during the Spring o£ - 
1976 but this was attributed to the hiring and **breaking in'' 
period o£ a new search analyst . It was also noted that over- 
all satisfaction tended to decline during those periods when 
demands for searches reache^i peak levels of 55 or'more per 
month. V .4 

An analysis of trends • since the employment* of the second 
searcli analyst reveals two interesting features. First, the 
previously observed decline in satisfaction levels with the 
Value of Bibliography and Materials during hectic periods . 
seems to be continuing, though with less frequency. It should 
be noted, however, that this ''decline" in satisfaction is a 
relative thing. That is, the satisfaction level for ^this item 
never fell below 2.0 be tween October 1976 and October 1977. 
This means that even at its worst, the service was never seen 
to be below a medium-high level of performance. 

Second, the Quality of Service has on four separate 
occasions (September 1976 , November 1976 , April 1977, August 
1977) been assessed at 3.0; the highest level attainable on 
a scale of three points. During that same-0cto6er 1976 to 
October 1977 period Quality of Service never f ell . below -2 . 7 
and hovelled at the 2.8 level. These sta'tistics ' confirm that 
the . perceived performance of the search analys-t is critical 
in maintaining high user satisfaction with the search service. 
It also points t(^: the fact that the ; user is, able to make dis- 
tinctions between the~ service he received from the , analyst and 
the final output of the system, i.e.j the bibliography and 
materials he receives. It also - seems .reasonable to- assume 
that he may also, be ..abje and willitig to ascribe hxs ^g^sser - 
satisfaction with his bibliography to; factors beyond the , . 
immediate influence or control of the search analyst, such as 
appropriateness of atopic, extent of materials available, "or 
system features. This ability to separate and distinguish. 
between the pe^rdEormance o£ . the analyst- and that of the search^ 
system-is .important because- if suggests that a user will be 
more willing to place anpther search on a different topic if. 
he feels that, everything 'humanly possiblevwill be done ^to meet 



' Satisfaction ;o£,.EIC Versus Kon-EIC* Users 

Because the human element of relaying and using infor- 
mation was, from the very beginning of this reisearch project, 
regarded as critical, it was felt that some special provision 
had to be made fo-r users who were located at a xiistance from 
Metropolitan Toronto and thus did not have immediate access 
to the service. The basic assumption was that the provision • 
of an 'intermediary , called an Educational Information Consul- 
tant, ar EIC, familiar ■ with local .conditions ; and needs , would 
lead to a more satisfied user and' more effective dissemination 
and utilization of knowledge. 'In turn, a satisfied user, 
according^ to adoption of innovation models, could be expected 
to become a repeat user who would have the obvious effect odF 
ensuring the maintsen^ance and growth of the inf ej^gna ^tion service 

To test whether or not such assumptions co-uld x)e proved 
empirically, special treatment was given to the Northwestern- 
Region of Ontario. There, four EICs in different types of 
^institutions were -trained , demonstrations of the service were . 
held, and reduced rafes were ins tituted fjbr the first year of 
the service's operation.. It was expected that this greater 
effort. would result in a disproportionately Jarge number of 
search requests from this -area.. - Table 20 below ^gjarbws- that 
this effort -^as successful, at least . initially . In the fixst. 

■ X ■ ■ • * . / . , '^ . ■ ■ 

. ' TABLE. 20 ■ 

; .-USERS LOCATED IN NORTHEASTERN 'REGION (REGION 3J ■ ■ 



Percent 



•Response : 



oar 1 Year 2 Y«ar- 3 



Yes 15.9 5.4:: " ■ ' : 3.9 

No ■ . SejJ/^"' - 94.6 ■ ' ' ' 96-1 

■ ' . .- as - [ 

Total- n- =" '380 ' ■ 371 .' 129 



two years, the Viegipn, with only 4% of the: province * s eiiroi- 
'in6nt in elementary and secondary schools , was overrepresented. 
The decline in the second year , from 14% to is due in part 
to the exhaustion of free searches . The 4% usage level in 
the third year is in keeping with the proportion of the area's 
population* 

Of these users, however, not all went through EICs. 

Table 21 shows whether the user went directly to the search, 
'analyst or through a local intermediary to have his -search 

formulated. The reasons for the decline from almost 10% to 
"2% between the firsthand second years are uncertain but may 



V TABLE 21 
WHO ASSISTED IN FORMULATING SEARCH REQUEST. 



Percent 



Role of Assistant 



Year 1 



"Year 



ar^ 



T 

97.8 

,0.8 
0 

'■ 0 

0.5 



Year 3 



Search Analyst 
Field Centre EIC 

Faculty of Education/GAAt 

EIC...-'' . ' 

Ministry -af EducationVEIC 

Not Siire _ 

Other 



85.9 



7.3^ 



0.8 

1.6 
1.3 
3.1 



> 9-7 



Total n = 



382. 



371; 



97.7 

0 

2.4 

0 



> 2.4 



0 . 
0 



12i9 



're^flect , among other things > trhe exhaustion of "free sejarches 

• the area, or . simply; a poor return of questionnaires.- It 
is also worth noting that these statistics do not take into 
account the activities of the EIC associated_with a local 
school board. Nevertheless, it wa^'' later found that 61 users 
in the Northeastern Region had/used ETSO. Of these 61, 18 
had returned questionnaires showing they had 'gone through 
EICs, while a. further 4^3 had not gone through^ EICs . \ . 
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Table 22 shows 'n:he means ,^ standard deviations , and fre- 
quencies for that sub-sample of^users who returned their 
evaluation questionnaires. Though statisticar significance 
is important, it is not, however, the sole criterion. for ^ 
determining the importance of a relationship. .Particularly 
when one is concerned with policy decisions', the stre'ngth of 
a relationship can indicate its practical importance. In the 
present case, a difference between means of 0 . 2. on the three- 
• point satisfaction scale is considered to be of no practical 
. significance Differences of 0.4, 0.6, and 0 . 8 or greater 
•are considered to be of low, medium, -or high practical sig- 
nificance, respectively. ;Due to the small. sample size, 
analyses of both statistical and practical significance are .; 
given, recognizing; that where only the latter obtains , further 

research is needed to confirm "-the existence of the relation- 

" • . ' ' ' ' 

ship. For the convenience. of the reader the 15 satisfaction 
items are grouped into the same five -categories, used earlier. - 
In the present context , the five main groupings in Table 
22 might be : regarded .as correspondirig- to the stages that: a 
typical user would go through tc5 jDlace- ,a* searchi . Fijrs t, « he 
would .obviously have to. become aware o^ the existence 'of the - 
service . Pijblicity Materials" and Directions covers ^ those 
aspects' that ^explain the technical ;poiTits about using the 
service to the user". , These ins tructions were -available in a 
variety -of written: formats and appeared as articles in *pro- 
fessional journals, advertisements, and brochures. It was \ 
expected i though ,^ according to .^the liter^ature , that personal 
contact,, either; informally or. through staff meetings arid- v/.p- 
planned presentations , would b^e"^ the most ef fective method of 
informing users about the service w And indeed,* the f irst 
goal the ET-Gs -were asked to achieve iii their initial training 
Has to make users aware of their availability -and the se^ivices 
they could provide. Therefore, one would expect the EIC users 
to be somewhat more satisfied with -th|6ir information about 'the 
service than non-EIC -users who were 'not expose^d to such 
personal efforts. - • There is evidence to support that this was 
in fact the case. 
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; MEANS, STiM Dmiia^S,^ SATISFACTION OF. EIC VERSUS NON-EIC USERS 



Aspects of EISO 



Micitij Mtemls and Divections ' 

■ ; Accuracy of ;publicity materials , EISO . ' 
•. • presentations,: etc. r . ,. 

Comprehensiveness of publicity mterials, 
"SO presentations, etc. ' ■ ' . 



, Adequacy of directions for submitting search" 
requests to EISO 

Adequacy of directions for ordering copies of ■ 
materials . ,. . ■ ' 



ElCUser 



s n 



5 ' 



1.1 0.52 6. 



2.5/: 0.76 • 8 ^ 
-2;6 ' 0.73 ' 9 



Non-EIC User 



■X 



■ s n 



2.8 , 0.45,' 5 . ■ , 2.4 0.79. 12 



■ 7 




2.2 0..67 9 



2..3 0.76 . 12' 



2.1 0.90 " 18 



l.'Ol 



1.89 



0.55 



1.65 



ittj of^ Service .., >' , 

Convenience .of arrangements to obtain .EISO ■ ' 
sephes . ' 

Helpfulness of search analyst or Educational 
■ InfonnatiGn Consultant ' v 

Time devoted to search intervi'ews with , search , ' 
i^alystor Educational Infomiation Consultant 



2-9 133 ^, .9 '2.6. ■0.61 ■ 20 . •;2.^ 



2.8, 0.62 ■ •12'' 



2.2 y4, .;' 5 



2..6^ 0.62 16: . 0.28 



2.4., 0.79 13' 0.20 



65 



■Aspects of EISO 



ElCUser' ' Non-EICUser 




Timelim^ of Service ■ 

:: .Tjie taken to deliver the El^ bto^^^^ ^ , u\' m jl - 'n .^^^ ' : 
TiiiB talcen to, deliver inicrofiche or paper cop^^^ ' 



7alue of Search Out^t- 
■ Value of bibliography 
Value of materials 



fm taken to deliver mifciche or .paper copies 
ordered from €ISO ■ \ 2.3' 0.58^ 3 '0 55 



<kiiitij of Teckohgij', . , ' ,'/ 

■/length of biMiograp^^ , g ^.2 

. feadabilityormicrofiche'cop^^^ 2.5 0.55 d 

■ AvajLlability of microfiche readers 



17 

2,1 0.69 7 
2.8 0.44 9 '-2.4 0.92 8 



2.2 0,79.- 10 ■ ';2.3 



16 



.2.5 0.55 , 6 2.2, 0.83 •• 9 



Looking at -the tour items in Table 22 that apply to 
providing the user with Publicity Mater ials and Directions, 
we .find that, measured on a three-point scale with three 'being 
the highest level achievable, the mean satisfaction level for 
EIC users was consistently -higher than for non-^EIC users. For 
accuracy of publicity materials it was 2.:8 as against Z-4; 
for." comprehensiveness of publicity materials it was 2:7 as 
ag-ainst 2,2; for adequacy of direct ions ..for submittiirg 
search requests to EISQ it was 2 . 5 to 2.3; for adequacy of 
...directions for ordering copies of materials, it was 2.6 .as - . 
compared with 2.1/ ^IVhile it is true that these figures, are ' 
not statistically . sigriificantT^, all of these differences are 
either of low or medium practl.cal significance'. . . Such -a conr 
sistent relationship between the use of an ETC and higher 
sat isf auction is of interest to the policy -maker pending con- 
firmation. ^ . ^ 

Since one of the main reasons for having EICs in the 
Northeas terrt Region was to improve the. "accessibility to the-^' 
information, service fo'r - geographical ly remote' usejrs ', 'one would 
expect EIC users ^ to be ^itiore . satisfied with -the Quali ty . of . -/^ ; - 
Service- than- those' who -did, npt -use 'the 'service' through a-^local^ 
intermediary, ' This did not* appear to- be; the ca^e however', 
since there was no difference of practical or statistical: 
significance between' the satisfaction levels of ErG'and'nonr 
ETC users with the convenience of arrangements to "'obtain EISO 
t^earches [2 . 9 toV2t6} or the helpfxilness -'o^f the se.arch analyst 
or Educational Information Consultant (2'. 8 to 2.6) . It will, - 
be noted that both groups of users were very satisfied with 
'these aspects of the, service and less satisfied C2".2 -and 2.4) ''^ 
with- >rie time de^a^ted' to the search in-terview with the searcH 
analyst or Education Information Consultant- Thais' lower 
satisfaction again shoving no practical or statistical 
differences. may be explained by the fact that, whether a " 
user goes through " an EIC or • directly to the search analys-T,. 
there are 'constraints- Some EIC users negotiated their searches 
"on the run^'because of their own job situations, and- thus the 
result may be' as much a reflection of the user's^ own ability - 
to take time -out for an adequate intervievg.-as the availability 



o£ the JpCv Noh-El^ais.e.rs may never-, in £act j .^ye"s^p<)ken-,tp 
the, ana-ly^t because "they, suliia^^ed. t;!^^ Tr ' .' ^eque sj^ Jfc)y^' 1bs&^ ' ot . 
.. if the'y- di.d- speak' to her, itvmay have been . ra't^hef .htS?^ed;ly:.V • . 

py .longr dist-ance tele;phQne ^^ "^ ;mode" riot conducive to leis'urely ' 
^prableni: defihiftiGn-"because :0,f -the • costs . involved^' . ■ . . ' . . 

^ • ■ : ^TThe are4s where one would . expect , the' non-^;:^IC use'r . to' , > - 
hav^ th^. advantage over the' E I C user are* t hos e that iBe 1 ate to - " 
vtime.. 'For the. non-EIC user would typically.'.receiye his biblio- 
graphy direct iry from the search analyst i - he' jwotiid^ ^^b^^^^^^ 
• Sible for order4.ng his .own. materials, an,d, he ;would receive ' 
these original documents in jriicrofiche'r^r paper- copy without 
the intervention 'of an intermediary -who .would represent '-e^- 
'otHer s^tep i-n the delive-irx^process . '.'the results shown £cr 
, the- items included under Timeliness of Service "^se em to- b.ear/ ' 
this 'out for one aspect: time -taken to deliver the EISO' .bib - - 
" liography, where the \ non.-EIC user has. a satisfaction level of 
•2.7 and the E-IC user, one of 2.1. This ' difference is great 
- Enough to. be of statistical significance as' well as medium 
practical ? ignif ieance . Since the other two items in this 
area^ Xime taken t<^. deliver microfiche or paper copies, from 
■EDR'S in the U. S^ , and time taken to . deliver microfiche or 
paper copie-s from EISO, showed ij^^ statistical significance 
-and. low (2:3 to 2.0) and no {2.4 to 2. 3> practical signifi- ! 
cancfe respectively,, the results suggest that going through 

• an EIC is influential at the first stage o°f the. delivery pro- ^ 
cess but does not affeot". the. later stages involving delivery 
of materials, from EDKS or ETS.Q1 - .; ■ - ^. 

I-rr the items related^to'^the Quality of Technology, there ' 
-is one that shows a statistical. and medium level- of practical 
•significance .in f avoi^ /of -^BJ-i^.users V satis'^action .with the ■ . ^ - 
length, of bibli'o^graphuy. This difference (2 .8 .to 2.2) might , 
be. attributed. to the greater . familiarity of the EIC with the- " ; ^ 
precise needs oT" has ci-ient. Differences between the • satis--, 
■faction levels on-'lsher' two othei:; items listed in-.'-i-thi s . 's'ejqt i ori.,- *.' ■ .-..f 
readability of microfiche "pqp-ies , and- availability of micror.- .. : / 
fiche readers, arte not statistically sigrii.ficaht and are of . 
■low practical significance 2.5 to 2 . 1. and. 2 . 8- to .2 :4 , respec- 

tXV^iy. , - ; .■ '■ .. •■ . 
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; ;■ {T-he filial -■stage, ^ x^^^^^^^ ass.ess^d byl' ' 

. .. . t^ie ;User' Eva.iuatio^^^^ to the'^Val-ue. o£ th^ 

• Se^i^ch Qutput^ , Th'e-; oiitput -re^err^b^ to consists, o£- two^^separate ' 
• "^V; products ;. . the ^iblio^^ materials ordered, - 

if .any , whrich- mght b"e. ei theT cojxies ,.p^ ; j oiirnal. ox' 
'■ microfiche duplicates of :ERIC .documerits or a; cpmbinatxoh o£ . 
the two. The mean satisfaction leyels for EIC , and ' non-rEIC .• 
.users with the value of the bibliography (2 . 2 . and '2 ! 3) and - 
\ the value of materials (2^.5 and 2 : 2) are so close that -the - ;. ' : 
-differences ..are of no statistical or practical significance.-- ' :\ 

V ■'.■:■:[ '■ ' [' ■. ■■ . • . Conclusion ■' ' z ' : -V • 

It would appear , that there is 'a slight, tendency for- e\[C . ' 
users • to be. mor4 satisfied with some aspects of the 'automated ' 
: •. bibliographic retrieval service than. non-EIC . users. These 

areas of greater mean satisfaction focus on the. 'pub life xty . about : 

• and directions for usi...j the service- and ihe quality ,/6f the 
technology. . ETC users are less s*a±is£i^d than their- Von-E I C • 

. counterparts wijth thos'e asp^ts of the service -that.^^add another - 
^ step to the ordering process and therefo.re slows down their 
/ receipt of ^the inf ormation . . This suggests thai the-yEICs role' ^ 
might be most, useful- in -the preliminary stages of -ra-isiiig • 
awareness and providing directions for using: the- service. It 
j^ould" appear-t.h§t the intermedxate stage of. search negotiation ; 
might best be lefu to the search ahaiyst and the..ordering of ' ' 
-\ materials to the user. . In that, way , maximum %Se- might be ^ade . ' 
. ■ ofr rhe local avail ablMity of the/.E^C, the .technical: expertise - , 
.. . of the ^^analyst; Vaifd the enthusl jieed of. -the user 'for. a '•>' 



mpre speedy delivery time. 
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■- . CmPTiER , 4 - 

the" NEGOTIATION PROCESS IN QNLINE 
BIBLIOGRAPHIC RETJIIEVAL - 



". . -• . ■• ^ •. ' • . .. ■ ■■ ■•• .■ ■•, ■■ -.-^ : / _ 

o£ the most impbrtant aspects of online 'search se;rvices 
is the negotiation process that. takesV-plra^e between/ih^ . ^ " ' 
librarian or searc£ analyst and the user seeki^^^^ 
For it 'is during t%at encounter that the , user must maike his 
needs knovm, the search analyst must comprehend the request, 
and translate it into the terminology and configura/tian that 
.will result in the successful retrieval of appropriate cita- 
.tions- . Yet little has been written that attempts to analyze 
what happens during the negotiation process. The purposes 
of :th is chapter , thVrefore, are to. review the • litbr^ture rele 
vant to the negotiation process, to identify the major • 
elements of that process as • outlined in, the: literature,;- to"- 
^PPly these theoretical- components to actual ^ negotiation ' v 
interviews and ascertain- their applicab^ thereto, and to 

develop a model - that describes the .component ei^ements: o'f the 
negotiation process. The development of such a" "model is seen 
as; having, implications for the training and' perf ormance -of 
search-: analysts ^and therefore contributing to" thes improvement 
of automated search services. ' ^ - ' . 



Review of*the Literature ■ 

Though onl ine s ervi ce s have .now "^eeh in ■ exi s t ence £o : 
more than a: j^e cade ^nd their growth 'and -e^^ have become 

the subject of scholarly attention (Fry 1972; Wanger et'aZ. , . 
1976), a recent survey of library science publications has / 
shown that little has been. -written on the negotiation process 



. betvfeen the user- .seeking^ iTii^Ormat?^ and the search ai^aiys t 
Working at the computer teTmlnal .v Whiie this newer field has ' 
remained unexplored, however, \ there exists an abundant litera- 
ture dealing with the mord^ ti?adi^onal ' form o£ the user/ analyst 
interface; the'. cla'ssic reference Hntei^view: That literoatufe ^ . 
-. ^^^^t^'S as its ma j or concerns ^ empathy and non-verbal behaviour ; 
the; interview process itself ,'^includin-g . its structure ' and 

• thel thought processes Vinvolved in the negbtiat-ion and formu- 
.lotion of ^questions ; ^the .Examination of . the establishment and : 
■ enactment - of roles by "the participants:, and other faj^toTs' 
which affect the interaction* between th^^ref erence I'ilxfar'ian 
and tl^e user. It- is to the^se^reas tl^- we now turn Jfor the — 
insight they can - lend to the online negotiation process . --- L 

. .Empathy ; and Npn.-Verbal 'Behaviour . - . , \ ' • 

' ^ . . Both Peck (197.5) arid Pierce; .C-197.1) ..have /taken the view.- -.^ 

that there is a strong affinity between -the' interviews that • . 

.. are part of reference 'services' a]|;d, those that are integ^'al .' w ' 

to therapy and counselling. They feel, therefore,, 'thaf an - - - 

awareness; of the techniques; emplbyed by counsellors would be ' 

beneficial toVlibrarians . in enabling : them- to improve their ' ' ■ 

services. -. One such technique is empathy. Peck -quotes Delaney . • 

and ]&i serine rg,;.twho . describe .;em|^ ;' ■ . 

'-■".-y:- • . the counselor's ability to urit^^^^^ 
-y. hpw the client ' feels ^ at any given moment ;iri' : ; ^ 

^ ■■ . r •'. > the counseling p.rocess and how -he f ee ls-*about ^ - 
^- : \- \, . the situations he is ;describing^ to l?he cburi- - * 
. ■ ■ ■ . - ;:Selor. Empathy also refers to the abi-I.ity -.^ 

; : f ^ to the client . 

. \ - ^i^" "•as* t^^^ communicate that the nature • and .>,' ; ' 

" intensity of the emotion' .experienced,r;is ^ ^ 1. 
, . ' . - " understood by the" cou'rt^elo'r fPeck , 19 75,-- ' ■ 

- p.- :2pi?. :: - -^...: ;^; ^ 4:..., . ' 

Applying is to . .the reference librarian. Peck suggests that 
empathy can be shown to- have e'ffects .- in numerous ways : the 
demeanour of the librarian at the initial interview can\ 
reduce the anxiety of the ii^er-; maintaining eye contact^ 
smiling, expressing concern can , diminish nervousness • .prac- ' . 
tising a relaxed,' slightly, 'forward-leaning, body position can . . 
reflect interest. .Other non-verbal behaviour such as nodding 
the head or. gesturing with the. hands can indicate awareness ... 
O 5 attention. , . ■ " 



. ^ V :> y j adds another dimension to^ PeQk^ s - article by: sug- 

: ^ . ^ ♦ • •. the import^^e of empathy does not • 
; ' : ' . .de^^end exclusive^^pon the empathic ability ^ 
r ; ' : ; ; ■ o -but a.lso on the^ percep- ' . 

; - > ■ V tion of empathy by the '^client (Pierce, 1971, . 

■■■ P.-..12.O0. ■ ... . ; 

, . - ysing- sijifcty male volunteers' ^enrolled, in an under gTaduate.-: / ^' 
J . course in^syc^hology at , the Ohio St^te Undversity , he carried - 
/ ; ou^ an exp^rimetit .to. testr the relationship between 

and perceived , empathy^ He subj ected the students to what he 
termed an "AppropXviate Timing Interview" (where the^^.^terviewer , 
> 'attempt e^^^ the students witlf^a 

" ^s?ratement\ d^id ^not inter ru^^ .student or allow nore than 

; .f^ve -seconds to .elapse) , and an ''Inappropriate rining Inter- . 
view.'-' ''^ TheClatter was dividied into' three periods: the firs t 
period of two •minutes was similar to the "Appropriate 'Timing ! 
< : Iritervliew the second was marked by interviewer interruptions 

. with a statement every five r seconds for five minutes; in the 
I third period (fiye to eight minutes)/, the^ interviewer did not - ; 
respond to tlie student for a fixed. period of twelve seconds- • ^ 
af^er the ^last utterance of the student • PieTce found that: - 
although the timing of statements was not a variable there 
was an inverse relationship be^twe en/ anxiety generated: 1^ the . 
■ :"^Inapprpp:riate Timing Interview", .a^^^ perceived empathy. The 

: ^ implication for. reference librarians is that perceived empa^thy 
. wo,uld ^sex satisfaction with and an improve- 

- ; ment^fN the li^ services. > ' ^ 

. / ■ ,Iix a somewhat similar experimehB^ r^epor;ted in detail, 
Matara^zo and Wieris (1967) . hypothesized (and 'validated with > . 
reported p values of .001) that .if the interviewer waited - 
^longer (or shbrfef ) than usual .before responding to the user^s 
• last c comment ,. the effect would be *an increase (or decrease) in 
' - the user' s own reaction time to the- interviewer ,. Citing other 
. t similar studies, they extrapolated that when; the interviewer's 
, . "^speech latency" (the reaction time before responding !to' the 
user '^s last utterance) was short, this, togethef\^ with head ; 
. . nodding and "saying "mm-limm, " acted as "social reinifbrcing 
. p ' stimuli''\ a:nd . led to the perception of a jnore positive attitude ,- 
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- by rthe interviewer,' and hence greater satisfiactibn oft the. p^rt 
of the user with the interview. •-; 

T . In another experiment .to test the' effect of the^ quality 
^ o£- a reference librarian' sV. inijned-ia:te. arid non- immediate verbal-' 
nonverbal ; communication on user satisfaction, Gothberg (1976). 
■ advanced three hypotheses : immediate rather than^ non- immedia.te 
ve.rbal- nonverbal comiifEinication by the librarian would result * 
in (1) greater user satisfaction with the reference inter>Yie'fcr; 
£2) greater satisfaction with his owri performance in. the hego- 
tiatioii of the reference question, arid (3) great;e^ user satis.-:. 
- faction with' "tlie l^arisfe Using the iininediacy 

.principle emerging f^m the ihve^ti^tions of Mehrabian arid 
^ his co-workers , a measureonent of the construct -^'satisfaction** . 
- . v.:used ^by^ V Kaam, and an ANOVA computer program ■ to analyze^.the 
<^..:-da4:a, she .foi::id that the first two. hypotheses were supported • 
_-by the reSxults V but -no-t the third . She ^s-uggests that mor^ 
: fx^querit immediate, verbal'^ communication will ^^crease 

the" le^'ei," of - the ylibrariaiT' s interpersonal functioning, ease 
the . frustration us^ eilCQun-ter in their;- search for infor- 

ihatiori, and :henc.e inrcrease their - satisfaction, with: the lib-rary ' s 
/.seryicesv-;.. , / .. ■ ^- Z /v'": ^ • . ■ ' 

rBoucker .(1^7 6,)/ also; ind^^ frustratipri 
exper^ienceid. b and, emphasizes the importance of 

. -rionveijbal communication in .alljeviatin^ this* She puts users 

• info fou'r'-basic categories : /the ske^ep., the i.vate pfztron ^^ ' 
the habi^tudl use^ > . pon fidenoe. pevsoni fi^edy 

that . librarians , ins^tead .of being in the "Preoccupation Mode*^ ' / 
should find themselves in the "Availability Mode*' more often 
if successful communication is ■ to take, place. She defines 

* the two/mpdes.as follows ■ ^. 

.\ . .PTyeooaupati-on Mode * AvatZabi^.Z-tty jMode ' - 

Arms across:^chest ' Arms relaxed,, not covering up body 

• .^Downward gaze so no eye : Eyes ready tOp contract those who^^ * 

contact possib-fe / . ente^ ^ 

Frown of cojicentrat ion , v Smile, of greeting, relaxed inouth, . 
firm mouth ..' . eyebrows lifted in attention. • ^ 

. .H4nds. busy with /b>66ks , * Hands, relaxeci^^ ^ 

\ paper, pencil , .terephqne 

Turned or., leanings a^ Leaning rfbrward, head up 




^^ad > bent down 

ERJC hind desk ^ - Willlngness..?tp :lea d^sk 



; To' coiit act. and iii prm . the us.er . sue ce 5-5 fui ly , t^he r e£e - 
ren^e librarxan ne^^^ skilled in- \^h^ art o£ conmunication, 

the nonverbal aspect b£ Which, though extremely^ imp'ort ant, < is . 
of tjeh 6ver2ooked. ■ ' . . • " - • : 

Structure 6£ Neg6t.ia^ - / ■ " ^ • ^ • 

" ; . Although very early in her;.article Boucher ^ does 'make ; | 
mention- of the tripartit<^ nature of the negotiation process * • : 
itself; it begins. -W^ moves Into in te if--' 

action and concludes with an infoTmat atosurB .she does not 
develop /this perspective in her article . V Gothbbrg C19 75),^; ; 
however V convinced . that . the negotiation of the reference ' 
-: ques tibn is one of the tia^ ©r •^problems .in ,librax^ ^:Commynica-^^^^^^.V^ 
: . ti-<?n,. believes that thje splutic^^^ of - - : 

. ^thi^'* nature *o£ the int^rp'ers"onal piXKiess , particularly from 
yth.e' per spec t of 'Martin BUber. Tiirhing to his view of the ^ 
Ampprtarice of > inter p^e^^^ , after an e'xamiha- ^ ^ 

tion of the" literature' In linguistics , Icinesic research, and / 
inf orma-tiTon - and trans actional theories,, she notefs that /'a few 
things^ stand outV':, the attitudes and .fee^rn^s of the helping 
persoii - are : more -imports than his 'brocedures br techniques , 
and ,the wa>^ in' which his attitude and feelings are^ perc 
maices the , crucial, differ enc then cited 

as f induing , in his review- of studies dbi^e /ylfnat. ease of access . 
to information is more important to the/seaifcher than th6 
quality of that information. "^:V 
Taylor '[1967} in considering the ,negotiation**^rocess , 
states his concern 'with two phases of this Interface between 
library system and ITbrary user arid has advocated -a more 
, dynamic rinouii^- oriented^^gpjra^ther. than a static (ob-ject-. 
oriented) ^ approach to the structuring of the' interview. He 
suggests thWt with regard to the first phas'e^ it is crucial 
to attempt to. understand. what a question is, and identifies 
fourv-;levels or stages of quest ion formulation. He however 
cautions that these Teve.ls are not separate, but . overlap and 
should be thought of only „as:..conyeni^nt/^p^p^^^^ a con- 

tinuum.. The levels; are;' (1) ^;t^^^ but; un*e;5cpr need. 

-£ox; i.nf^^ iithe^ ■^'Visceral need) ; ^2) \:the cons.ci - " - 



■r ' withi^rbrained descriptibri of the- need' -(th& obn^oi^ous need;) ;, 

(3) the forinaj'.: statement" of the need ; C t he - forma-Zis e dViiee^'; ;. >■ 

(4) the question as presented to the information system' (the 
aomp2*om'Csed need') . - 

-/unless the inquirer ' knows . the infx)rmatiori. 
-^specialist well, he is inclined to pose his .. - 
.■ • ' , • fijrst -question in positive and- wel-1- defined .. 
terms, ,even to the point -of specifying a 
particular package [level 43. . .^^^^T^ 
- slcill of -the reference librarian- is to work • " 

• " . * ■w'ith . the inquirer back .to the -form'ailise'd V 
: .- need [level 3], possibly even--to the con- - 

' s-ciqus-need-^ lj^ever .2 J i^^ to - translate '• 

'. . these , -heeds into a useful search strateev ^ • ' ' ' 

, CTaylor, 196 7 / p . ,93 . " • ^- .^^^^^^ 

. : . ^ The second phase of ■th:e rpro^ess • of rnegot 

tion is the .identification. orf /the "filters'; thr^^ which ai - 
■question passes. From interviews '^^rith librarians ahd infor- 
mation . speqa^ljL;sts , Taylor pinpo^^ %liy :sxxbjetZ defi- : 
'^ition ; ; (-2) objective ■:and motivation of the, user ; ./(S)- . persbnai 
. characteristics:^qf~,^^^^^^ C4> relationship of intjuiry 'de^^^ 
-!cfiptioii to file .organization; (5) anticipated ^qr acceptable ' 
■.'answers^.-."-.'' ' "' ■ , ^ ' " ■. •'V'?- : 

The first "filter" is concerned with provided, the 
librarian with a general delineation of the: ^ubj ect area- o^^^^^^ 
. inquiry through feedback; the second, which most librarian's ^ 
felt was- crucial- to the success of any negotiation' and conse- 
quent search, stems from the observed fact that while users / - 
.often cannot define what they want , they can^discus^^ w% ■they 
. need it.; This stage further qualifies the subject .and helps 
the usei^'and librarian . formulate a abetter understanding and . ; 
clarification of the user's n-eed. The third, the personal . . .;, 
background of the inquirer, according to Taylor, . 

. ' . . may well . determine the urgency,, the - - ■ ^ 

- strategy^- of the negotiation,, the level of ■ -"^ ' 
any dialogue. and the critical acceptance 
. of sear.ch- results C-Z'2?icf. p. 14^. 
^In short, it" Is . the ' context" and environment of the- entire • 
process . Then , the reference 1 ibrarian translates , : inte;rpreis , 

^:^.;f.^^^<="t^^"es it. fits the files as 

'they .are. /^organized in the- library "systeiri" /: Final ly , th^^^^ usei:-. 
^•Can be made aware of the capabilitiesof both the;>;il>rarlah^ 



v^ . and the system, adjust his..- seanch strategy in ^the- process , 
: . :'Pj^C6- limits of time and size on his inquiry and possibly - 
alter his expectations o£. acceptable answers. 
\ 5- . . -Taylor sets .out to s'hbw tjiat the negotiation process", 

. . dn its best form:, is structured and can be analyzedi Onc-e 
:-the St ru^^^ are understood, they can be; delated 

.. (and even, further broken down) for fruitful analysiS^and the 
eventual improvement of reference librarians and information 
•specialists- This would lead t<r a better understanding of 
/ , their role and therefore greater efficiency and, effect xven^ss- 
in serving their various publics. -° ■ * . 

^ ;<)peh . and^.^^^^^ : ^/ . \ t. . . . ■ 

": • " • '-^ ^6 2&11 ^^^^^ a fairly " ' 

_ .accurate notion, of what happens during the .r'e ference-.procesSjl 
- - - ■ there ;^is-. little q^- how %t. happens-. He. suggests that a ' 

knowledge- of the. thought processes . involved .in recognizing , 
.•^ types of questions being asked," in formulating, search strate- 
; ■ ■.> ;gies , V and in selecting\the ld.nd":of answers r^^ in 

reference work i's needed to "improve , the teaching and practice 
. • bf ""refer One model that he feel parallels this'- 

:^ process is Guil.ford' s Intellect Model . . - . ' . " ' 

An integral part of this model is. conve'rgent and diver- 
: gent thought prodiictionV ."the bperations most relevant to >. . 
the reference process" (p. 310). . In convergent. ..production 
. opexations i logical necessities are generated from given ■ - 
information, and in divergent production operations, .logicai 
pos'si^hi^lCt-ies.. 'llPiae^crucial differentiation -of : the: two kinds 
of operation is that- tJie kind of response is completely speci- 
fied in -convergent prbdudtion van4 riot in divergent production" 
, Cp- 313) . Fluency, f legibility, originalityy- and^^the ability 
.; . to elaborate, de.taii are abilities . associated^ with the f ormer, " 
while the drawing of deductions ..that are' logically ti^ht from 
given -inf'Qrm^tiOn ^^^^^^ are essentially, uniquely 

: : deteriTiinQd^by the* given information): cHara.c^^ \' 
..- production ability-."' The "reference .1^^ able" t'o 

■: . . .engage, in. both forms and'miifst' therefore be., aware*' 6_f:~^^thje .; 
^-kl-thbug^ involved 'ito emerge^ with a better; def init'ion." - 

' of the problQm'V . ' - .- . . - " ■ - ": . 

■ ERIC :/ 9{p-\ , - ' ■ - ■ ■ ' ' 



In concrete", operat,ional^ terras these two types o£ 

thinking are what .^ing (19 72..)' has categorized as ".open" and 

"closed" (fuestions, Sh'e advocates that the understanding of. 

the ''userVs request and the supplying of the information- 

requi-red' would be immensely fa^cilitated "by a. more pion'ounced 

use of one technique -- the' use of the "open" question. * 

Questions are open when the response is left . . 
up to the respondent; when they cannot be " 
answered; "yes'' or "no/'. Example: "Wh-at is'', ' 
it that you want"^ to know about soft contact 
lenses?" . . . v * / , ^, 

Questions are- closed, when /the respond^t . ■ 

'does not have a^hoice in his response-"' ofchex 
*. ' than, those' provided by" the. ques>tioner. Two . ^ " 
• _-.' ;-general. types of o^s e d quest iblis ar^ 

' , • whi^ch may .be-^'ansvere.d^. ."yes"..or "no" :aiid . 

/ -.those; which are^^^^^^ Examples: ■ -^^--^ 

^ . ^ -"Do you want , .a: biography'?'^ Cy^s or noJ) • * ^ 

"Do you" wantf books , or informatipn?":^!^ ' 
choice:-, books or .information] . (King , 1972, 

wrds that preface "open" vque^tioi^, are : * '/what^^hen, >how,- 

who, where," but the most 'important ■ is ."why • " * For "closed" 

• ' . • ■ ' * , • '. " - ■ 

questions, "the tools oF cro^s-examinat\on,^^ ^'i.s , do, can, 

^ ■ .' ' ^ * . ■'■ ■ . - ■ • • ^. . 

will"' are usually the words at the beginning ^f the question* 

'On the major -assumption that. good jref^rence librarians have - 
to-be good interviewers, 'the "op'en^' question, which encourages 
the user to talk and a^Xlows. him rather than the libraria^n^to 
shape the request, is of 'prime* importance to the successful 
conduct- of the negotiation interview.- 

- ■ . *♦ . -. » 

Rqle-and Status Relationships 

The importance of ^understanding the part roles play in 
the comjnunicatipn' pr^oce^ss is another area that has .Tecei^ 
some atteritiQii. ■ Gothberg. (19733 states" than in "ail o'rgan^i'-: * > 
>sation 'such as a library , . there exists a- structure of roles- " 

tied together by lines of communication" (p. 13) . and laments 

' ^ ■ ■ ' ' ■ ■ 

the fact that the present situation is' one in which neither 

the role of the' library nor that of* the librarian is clearly 

• ... . ■ . ^ ■ ^ ^ ^- ; - y / I 

defined. .This resii;tts in role .conrflict when" the intumbeait 

i3- notable, to^'fulfrll rc^le" exj^^tatijon^ realistical is 

unhappy with his image of the role, or with fii^S^^rceived 

image of .himself in that- role . \s , * - ( 



. This, view, also finds suppo-i;|t in Shosi-d" (1974) ^and Horn , ^ 
^•(1974) .-■ These authors take^'"^ive position that: neither the 
librarian nor the client know^ what to expect .frpin each other. , 
As\a ' result o£ the .coiifusiorr of their , precise roles , libirarians 
have .o^^^^esorted -t condentipjis of . users as "patronsy^'- and 

therefore patronise them,, giving them what they- want regardless 

' . ■ . i ■ . — ■ • • . _ . ' • 

o£ other issues involved. ' . 

• ■ . ' ■ ■ . ' ^ y - ' ■ •■ 

: ' According , to Shos.id (1974) this confusion is f^sJierecT^ • 

by the open-endedness and unscheduled nature -o£ tjie* reference. 

-encounter/ the, lac^c pf control on the? part of the librarian, 
and the concomitant need to establish relative status^ The.: 
author claims that the reference librarian^ is. ■ thus placed in. 

'a. situatipn^ whicla^ Goffman describes * as leaving 

. \ . the performer in a ^ppsi tion pf not . 
' knowing what character he 'will, have to 
, pro j ect from one . moment ta tjie -next , . * • v .'. 
- .^ ' making it dif f Lcu!^t--fpr him. to eJEfect a . - 

. \-dramatdfrgical succes^ in any one qf .them . * 
. - (Shosid, 19 74, p.. ^20). f 

•;Shbsxd* also sugge;sts tha of. the^ user is another vari- . 

abi^^ in both' status and 4eference patterns . in the. .ihteractiohv " 

In short, misidentif ication ,and/or disagreement as. to what 

the role is influences the j udgement ..bf* the .user regarding the 

librarian's adequsCcy^ his own satisfaction, and his decision 

. • ■ J • .• 

to use- the ise.rvice in the future. ^ > . 

The ambiguity of the situation, in which the reference 
librarian and user find themselves is also intensified by . S 
the disparate and often competing combination of functions 
that ^comprise a reference s^ervice. These include the demand 
for reparts and s tat^stic's^ by the administration who tend, to 
think of the organi zat iofi .o?/^ library as if it were : only • 
open during normal ^working hour's (Horn, 1974) ; the amount of- 
time . available to "negotiate" the request; and, most impor-.- • 
tant", the lack of ob j ective measures to j udges.ef f ective 
•servic^e both by : the: user who tends to' judge.-<it by th rela- 
tive status estabiished daring the -interview,., and the refe- 
rence. Jibrarian , who often receives no feedback whatever on 7 
the effectiveness of the service provided:, , . - •* 



■ Suimnary ."■^ \ • ./ » v - ■ ' 

Alt:houg]i tKere is as yet a general lack of material 
.treating the*' online negotiation process, there exists sonie " 
li.terature ttiat examines' the traditional reference interview^ 

*;The ma-ih aar^a^^ that literature . discusses have been considered' ; 
here. these were nonverbal communication-, the structure and 
thought processes which underlie the interviewing process,, 
and the establishment and enactment o'f the roles of the par- 
ticipants. It eremains^'^f or .the- following sections of this chap- 
ter .to examine how adequately th^se areas describe the online 
negotiation ■pro_cess- ■ • -T . 

■ - ■■ • - ■*'■'' ' ■ ' ' ' " 

. . - : - Design. :of •the;/Study •* ^ : ■ ' - : ' . ' . . \ 

* - - / There are:^ basically; two w^ys 'of generating understandings 
of what happens- during a given.'process. One method, is. to use. 
previous literature and .concepts that others have developed 
to- determine whether they explain what is going on . -The other 
approach ;is to . study the data to see if there are regularities/ 
"in; it that' allow one to ^identify nfew ideas that ^ perhaps othe'rs 
have .not/ not*^ . ' On - the one haiid, /the resea.rchef. is. trying to 
see -if ' pre->xist ing interpretations can illuininate or explain 
what , is "going' on ^^^^ process- On the other"^ ■ the research 

-is attei^ting- fo geh^r^te^^nev/ ideas or theories (Kerlinger'," 
-1973 ; Glaser.; and Straus>.^^^^ This study uses; both methods ' 

on the assumption that different perspecti.^^s us^ing different 
conceptualizations can lead tiD a bette^r Hihde't^tan^i^g of the 
process^ in question. Such an improved understanding, it is 
f eit , . willvcohtribute to mo*re \'ef f ect ive training programs -for 
search analysts- and 'better -negotiation interviews with 'uisers- 
. . This Vstudy ^is " mainly -an" exploratory .one . It 'uses com- --^ 
parative. anal^ o£^ negt)t iation^^interviews. witTi/ differ en^ ^^^^ 
types of clients order to ".maximize -the., variation bet^^ ^ 
contracting da-ta!.-' /Negotiation^ intei^views were. dic^£fe£,^jmiz^d 
^intb academic and nqn- academic .(s-ee Table* -2 3) rep^!esenti^ 
the .main roles of the . cl-i.ents 'of^the Educationj^^^ 
S>^tbm f or ' pnt^r io - V These- interviews^- were further categorize-d. 
by the s^ex;-Qf the ; user , f ol lawdng, -sugg.erStxonis . i^^the :i±i:eTdi^ \^ 
ture that ; different relatipAships.' mi ght^;^d a giveii j; . . 





^ISO DSEK SAMPLE SELECTION ^ " • 
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Characteristics 














— 1 



Type 
R^le 



Sex 

. Category Name 



"New^. 

9 

Academic^ 



Male 
A 



B 



Non* Academic^ 



Feinaie Hale 



Female 



rr 



. F-tZe ChcayzdteTist-ios 



Total ■ number of 
'£ile^ in each 
category . 



Number ,o£ files 
in each categor>- 
cqntaining all 
required ini ott-^ 
mation : . 



17- 



* 11 



1 6 



.rr. 



. 7 



'No complete data sets were -available 'for repeat users.' 



■e.g.-, OISE •• ■ ^ - . / . . _ . , . ■ ^ 

"'^e.'^..:^- Government- administration; school ^^ard meiribeV- . 
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reference librarian because of^the social identities a£ men * 
and womenv . Although ^the total number o£ uSers. in each' of^the 
£ouT categories, male and female academics and pf^n-acadei^ , 
wars 17, li; 9, afid 7^ respectively, substantially ^fewer- user 
files' "Contained' all the information heeded (transcribed . , 
negotiation interviews. Search Request Forms , returned User- 
Evaluation Questionnaires ) for analysis* The final selection 
consisted of one raridomly 'selected interview from .each of the 
four categories^ of* users for^whom comp'lete data was available 

Method -of Analysis ' . , . ... 

. in the review of the- . litera-ture^ major concepts relating, 
to reference i-titerviews were^ identified,"^ .It was felt that 
because negotiation ^interviews foY'pnline information retrieval 
could be considered similar in many "ways to the more tradi- . 
tional type of interaction between librarian and user, arid 
because little existed, describing the online process, the 
literature treating tlie older^ topic could be applied with' some 
justification to the more recent one. . 

Therefore, the first task undertaken was a review of 
the -relevant literature to . identify the major iioncepts that - . 
might be applicable ."^ The areas that emierged were :, empathy 
and nonverKal behaviour, the structure of the negotiation 
(includin'g 'the ; us'e^ of . open and closed questions) , . and role " - 
an\i statuS' relationships! For these concepts, the method^ - 
b5^^atialysis -was . a counting 'scheme to d'ktermine how frequently 
each Vas" used in the EISO negotiation interviews"/ After the v 
extent" to .which that variable occurred in the interviews was 
measured, -^an attempt was made to -.account for the different 
leV^els of occurrence.. 

- . '\ Description of Findings 

The Negotiation- Interviews . ^ . / ' . 

User A - ' •- ' • ■ ■. • ■ ; . ^ / r , \ 

- ■ ■ " — < *. • « . ' 

' A Master of Arts' student in the Applied Psychology, 

User A apprbached the search analyst iyi the early stages of 

formulating a topic for his master's thesis*; /.In answer' to 

I 



the isearch analyst's i^nvitati'on to describe ^in general^. -de-rms. 
what^h"i^^^ . , - -^^^-^^^^ 

* r "^U-A/Rli My study will* involve children o£ ^ . 
' ' : • - . low social and .economic s'tatus • ^ ^ . 

There will be 'two groups^, one eoh- ^ 
trol and the other experimental, * ' . 
' ■< The experiment al. .group , will have x - , 
'-^ . ' low self-eohcept o£ abxliity. . The- ; 

^ . ' ' . control group will have high self- _ . • ^ 

. . ■ " ; concept o£ •abdlity. . ■ 

At this point the search' analyst irfakes her first- interjection * 

and the interview proceeds as' follows : 

^SA/Q2: -^J'his is of their own abilxty / 
' : - * jp generally? - . • • ^ 

^--U. . -U.-A/R2 : ^ Self-concept rs how they s^ee " them- .- / ^ 

*^ ' - - - s elvers in relation 4:o ,the kids in -V;* ^ 

, ' . . ..^-^ clkss , smarter or dumber*.- My ^ 
, :^ hypothesis^ i3 that the children 

. ' with low se^lf- concept ^o^f ability . 

.- ■"^ ■ are more Ixkely to perceive a di:s- ^ -- 
. ' tance on^certain dimensions between 

. ': ■ themselves and" the professionals.^ ^ ^ " ^ v- 

they come in contact >/xth than 
, children with high ^If-concept^ of 
' ability. - 

/ ' SA/Q3: They' re more likeTf. ;to be - in- awe of 

^-^^'e^acKers and "doctoris and such? 
- ' " . • ^ ■ . 

r - U-A/R3:* They see a distance between them^ . ' ^ ^ 

- . selves and professionals* on ceVtain 

" • . dimejisi-ons . That's basically it. 

Even'' from thi§ brief excerpt, a number of things stai^i^ out. 

F.irit , the. analyst' s primjary t^sk ^is to define what the^ user' s - 

search topic is about. ; She ' does this by . as-kxng.. , a general o 

ppexi,- exploratory 'question which eliqits. an .overview of th-e ^ 

topic from 'the'Us'er (Respdnse' 1) . The :topi^ is a .fairly^ - ; _ 

sjSeciklized one ' and the search .analyst 'must, a^k probing, ' 

'defi-nitior^l questions CQ2, Q3} to/make siire that she, knows - ' 

wh^t the main: conceptual components '^f"' the topic are. Though 

the user may feel :that., he has' done an ad^uate job of explana- 

tjlon knd. may evenybe prepared to ."terminate the interview soon' . ^ 

after it has ^egin (see .R3) , the search . analyst feels the' need 

to probe furtlv^r before planning »her searc;h.r "Are you concerned 



■^User A, ^Response 1 • . . ^ . / ^ - 4^ 

'^Search Analyst, ^Question 2 - , / . ^' 



with p.rofessionars other 'than teachers?/' -(Q4> sh • 

. (Q5) she continues. She determines frdinhis answers /.that he 

is interested in doetors' and lawyers as: well as teachers and " 

that he will.be using a selfTanchoririg" scale • first used by 

Kil.p"a4:rick (R4 and R5) The analyst, queries the meaning of / 

a ^'self-anchoring" scale CQ6) The'' user expla.ins : - 

: U--a7r6V ^ What I'm going to* do.' is ^sk the ; ; . ' . 

— ' . ^ child to th^ink about"^ ^*-^b f or her ^ 
- . • - is on th^./^^op end of scale on . ■ " 

■■"'^ . . ^ < • ' the" dimension and. >wL s on the *■ . ^y -. '. ' 

; ^"^ bottom ^end. Then sh given r 

- ' people, including Hersuif, whom she - • 

■ r .'is to place on *the scale so tha^ 

' " eventually I'll find -out where the ... 

child sees herself . in relation to 
'^.^ t the professional . ^ • " 

At this point, the sear ch analyst reiterates t;he -topic in her 

own : WOT d'^ and begins to thumb through the thes^-urus -of -terms .. . 

used in Tsycholog-ical ^Ah^trac^s loo'klriz f.o^^ des- 

criptors.. As she does-so, she seeks confirmation or rejection 

of her choices through a series of* closed questions.- 'She "a^so" 

> ' , ■ . ' . ^ ■ ■ . ... ■ • 

begins .to 'exqlain:^>her -choice o£ P'sychdlogicaZ Ahsti^ao^s as ; the 

■ . - *' . * * ■ , . * . - ^ . ■ ■ ' 

data base she will search/"' describing its ^features and tho;se . 

of automat.eii searching in general *, "" As she proceeds to identify 

"synonymous terms for the major concepts .t"h*at1comp.i'ise the topic 

and suggests these to^he user, the^user continues to delve . 

with grt)^wing specificity into his. tgpi.c^describing the research 

design he proposes- to\ use. and ^he^yc^cAniq-ifes .he. wiir for 

analyzing his dat'a.- Thi-s inc>^Q^aSri.ng^ de't-a^ 

prompts the analyst tg ^^^ S^^e whether the. useV won id be in- 
terested ill ^ 1 on g^t-udi;^^^^' follow -lip studies. . Eurther 
narrowing of vter^s talc^^fe)lace\through- another .series of 
^closed questions - ^ The ^Vil^^r^^^ analyst ^ feeling t-Iiat. she . is 
coming to grips with . the '^topic ^ an increasing amount . " 

of ItijTie; e:?^plainii\gt'^e logic of search strategi*es and- the ^ /' . 
formatsV^in whicft.^i^scrlp.t.ors: m^y ;be entered at' thfi terminal . 
She pre^^aces ^.om^-^cf her remarks by emphasizing the limita- 
tions-of^^th^-ej^ystem^-T-^ concepfts you . \ 

ask- for 'are .very .TrS^rd; to- specify , ^ you may; get a . lot o"f 
odd. refeTences . ''^ The interview- concfu^ies:^ with the Search 



analyst telling the user When the search will be .run and /when ^ 

- - h e- may e xp ec-t - to. -re cei-ve- -h is -- b i b 1-io g-r ap hy — ;~ "-- - :;v -- 

From her interview with, thgr user, the analyst, construc- 
ted a search strategy the next day that combined . synonym 
terms for the topic's main ideas* The majqif, concfepti' included, 
were "self-concept,"- "ranked selves," "described selv^ 

- "ability i" ."career," "prof essdonals/' '^stereotypes and 
"role." . In all , . ninety.-one search statements were needed to ■ 
adequately refine .the tppic. At the end of . a week the user 
received a .bibliogfapliy containing 136 citations; He found 
much of the material interes^ting but. not relevant. - Neverthe- 
less, the user expressed high satisfaction with- th^ 'search 
scnalyst and the forty-minute .^negbtiatipn interview , he had. had ^ 

with her. ^ ■■■ ' ' - • - ' . - ■■ - ^ 

■ ■ .■ j» , " , . ^ - • " , ■ y ' ^ ' 

r ■ \ ■ ' ' ' ■ . ■ - . ' ^ • . 

User* B ^ v, : ^ . 



The secoi^d- interview to be .discussed lasted approximately 
nine minutes;. It began in a similar fashion to the- first with 
open questions eliciting the ^ general natui^e;.of the information 
request: • 

, SA/Ql: What is your' topic- about? : ^ . 

- ■ " " . ^ •■ . ^ . 

U-'B/Rl:: It has to do w^ith library services - - . v, 
in e lementary schools . -What I^m * ... 
. * _ looking for is: What does ' research - - 

\ ' . say, about the inf 1-uences of lib.rary ' 

: practices related to the" number of, 

; bobks^:Te ad' by kids?' . , ; ^. 

- / . ■ SA/Q2: ' IVhat sort of practices did you have " T 

7, in^miiid? 'r: ^- . ■" • ' 

■ \ U-B/R2 : . ; Well^ -at one time "I^^ 

-. .from the University' refer to a. study.,'.' 

\: • " or three, studies it might have beefri. /' 

.* ' - - In otie .case the kids- wen-t* to the . * ' 

' "* cent^i^l' ^library to -pick. -up books in ' . : - 

- r: / .another .caise the - teachers brought' the 

" V :t>ooks back to the rooms and left them 

.. for kids to take put ; in the third - 
.- . V / ^ . case, •the- one- that intrigues me , , ^t^^ \ 

/ . . : ' ; • . teachers" brought ^tlie books* to the - ' ; ^ 

. . . \ - . ^ room , created, displays , arid gave book » 
' .. . - ' . ■ ■ , ^ ":igLlks ^ As *I* recall ,. this last case 
. . ■ . resulted - in more .kids actually regid- ' 

- 'ing--^. 

.. . This is really related.' to tlie nego.-^ - . ■ ^ 

^ - ^ • y. ' • ^tiati^hs with our school, boardl. .The ' ■ . 

■ ' ' ' y- . ^ - elemehtaf;>^ teachers want to negotiate 



,tx> Have librarJL^ns de:S ignated -above . 
- - r at i c-~£o r -t he yS 'chob 1 '-s v-a 1-loca t i _-^^„-—v_ 
* ofv staff and Zi'm r;eally trying to 
V find out, .whetner we need schoo^l 

librarians and what alternative ^. . 

{pxactices we might consider. ; . : : ; v.:./ r' / 

::So you' re looking for way that j» ,\ 
'. money,- could bie sayed-a:f3d work dis- . ^ 
. ■ '• ' .tributed .a^ produce positive 

: ^ :^r ^l}^B:m^ 'That's right.: ^ . 

• '^ The content and reason for the search- having been arrived 

'■'^ at fairly quickly,- the weight.^'of the dialogue now shifts to the 

^^earch analyst who. launches into - a description of the . systbm.^^^^^ 

_^>^gain,/ she caut^ions the user about its capabilities.^ ^ '. 

. . SA: .^It's going to^ be -hard to' get - at the \ ; ■ : ; 

: ' ■' '^.notion of taking books . to vthe class^ * 
• ' ; ■ , ' room, ; I doh 't think therB are des^ 

V ' criptors for that<.*.[She searches 
,* ' ; through*; the Thesaurus ' of EEIC Des- ^ ' ^ 

. ^ ^ / ariptors . ] ; I ' 11 sSow you what I'm 

. . ; looking • through 

There follows'^ detailed' explanat ion_of - the lisg of descriptors 

. in formulating search strategies . ^ 

A less'bn in Boolean lo.gicjis followed^iy a rapid exchange 

' of jrclosed questions and answers ±n whi;feh-<the search analyst has 

the usejT thumb the Thesaurus., to suggisst : descriptors that des- : - 

6r.ibe ; the main concepts of his topicy. : -Shejconf irms'^^is* choice - 

or suggests new terms- . _ ; ^ 

' U-JB: Here we. go with "elementary school.^" 

' SA..: . That would.' be :.a good * term probably . 

v U~B: ^ ^^^Library programs?^^ * "Library role?" - . 

SA: .Yes\ ' "Library circulatdon is a .good 

ondv too, I think. . We';re not going to 
fina brie that> ca;n 
- y.. : : ■ put Idown ones that look use^ful^ 

, - ■ ; U-B : Th^re ' s ^^SGhool lib-raries .A - ^ ■ : . ^ j . 

- ^ "SA- , - ^ Sounds iCi^ "elementary ^ 

; ' . -< * school ■ lxSa:a;ri^^ might be better . ' 

... - - U-B: Ho3v about "lihr^^ 

^' . ^ ■ SA:: * - That, would go more with the idea of 
, V • a central library . I tiistink^ If it ' 

^as* done .in>. the classr^ and tiie 
teacher -was^the main person in charge, 
-there- /probably 'wouldn' t really be 
^ — * • -V ajiy thing major enough to be called ' 

• ' . * ; . ;*'"instf u " ^ 

ERIC' ■ ■ \- ..S5>- ■ y'V: 



,.^v.ing..e,xhauste'<d the ' library-related . qoncept , t: 
' ' "turns. tO' tSe ; definitibnVof staff '^designated ab' 



the interview 



o.ve ratio 



Wi th- tha't- defined , - the s earch analyst 'draws the ■interview' to 
a close by referring brief ly -to i the cost of the-, search .an^' 



assuring the., user she will do h^r utmost to see .b.e- gets^his .. . 

money * s..\worth. ^ ..:''■'•■< ' -V ; '•'"V''- ' 

■j- , , From the information "gained in the^<4nt-eT.vie.yf- .^^^^ 

analyst created a search strategy tha^^r^etri^ved ^^pCitatipris, ^ 
The user rated his satisfaction with jClie. interview, the. bib-^. 
• • liography.', and the original documents,'he :'.prdered as high and:' ■ ' 
\^appended. an additional com^ his- returned . ■/ ■ 'p]'^'^^. 

•' Yeyaluattri'd'h'.question-jfiaire \' " • ■ ^ 




\f SA: Gould you tell me som^'thing' about th^ 

- topic you're interested in? * 

' ' - U-C:. Basically, I want to know about com- 

; - :puter assisted instruction. 

SaV . Okay. 

/ ' U-C:/^' The siiccess or failure of it. • 

SA: :; -When we made this appointment you 

said that the foundation you are wi.th^ 
, is for ' the , physically han^ic3-pped. - 
tat right-? . \ . . " 



ERLC 




U-C: V It ' s j4»«.inly for cerebral 

p^^rsied children. 

SA: - Uh-huh. * ' ' " 

U-C:. ' _ Nbn- verfeal kids . 

SA:- . .Right.. Wculd you also be interested 
in information dealiiig with normal 

' • ■ /. kids^?,. .-^ ^-^ . - 



U-C: * - SureV But if . there J s . anyttiing on : . 
-;brain- damaged kids , that would be 
-better, v ; 
|S^;SA:: . .. .Fine. ' . ' ■ ^ ^, - 

" U-C:../ . Basioally what we're , thinking, of 
/ \ doing, is teaching kids to .learn their 

' • 'Sr symbols* by computer program!. - What , 
I'd like to know is- it, Sn fact, I 
' can* say that "computeriz^id teaching is. 

an improvement o*rer: ^other Jci^xds 6f 
^teaching. ' . ; , > - ' 

. Uh-huh. . r . < • ' 




.6 5 . 

' So 



U-C: . I need evidence to support thj&ir 
^_ goirvg into^using comp^ers_:^to teach 

" V . the s e IcTds; . ' ■ ' \ 7T; , ; 

■SA: \ ' I. see. . . 



:■]' 




U-C: The cost is the ma^'or problem.. They 

r can always ask whyv^a workbook or a . 
teacher with a blackboard \wouldn' tV 
be just 'as useful./ ' . 

/ . SA: ' ; . Maybe you couldi't me- a little ' 
V ^ - . , ^-mox^ybcyut .;this/^ymbol^ 

\ r ' ^ ; .'• being ^tatight . \ . ^:<.- ' ' /^-^^^^ 

So' began an interview dealing with the; relative strengths and 
weaknesses of us-ing computerr assijs tedvijxs'truction -f dr 
symbolic -language to childred affected by cereb-fal. palsy. ' 

_ ; vAf ter^determining . t;:e -ge^^ n.ature of the Atopic, thW 
search analyst -examinjsd each of the .major concepts in, turn 

symbolic language, computer^assisted instruction, neurolo-- 

. gicatly handicapped - - finding synonymous terms for each, 

expanding and contracting the'; scope of the concepts. The fol- 

lowing excerpt is illustrative: * " if 

] / *" / ■■• ■ \ 

^ SA: As an addition to the te'rms that have • , 

. '■'y "computer" in them, we could ask for 

Z^. 'stuff on programmed instruction. : 

V'C: ' ..- Yeah;' ^ . .\ ■ > ■ : 

SA: -It doesrt' t have t-o^'^be just computer 

* • ^ assisted,' does' it?' 

U-C: No.,-, that's the same thing tool * Iji , 

fact that would" be a good choice, ' . ' 
, . ^;^ctually, . • < > ; 

' SA: As a se^cond choice or an acceptable 

. >>: alternative, perhaps. * - ' - • 

U-Cr. MaybQ- we could get tha.t with reference' 

to« brain, da^an-ged kids.. 

^ .: - • SA: " _ Yeah. We coxild ask for this type of. . . - 

• • . system or this : type of kid and. then ^ . 
\ V . if we -get a small number:; [of cita- 

tions] back, we can broaden computer- 
^ assisted instruction by adding 'prb- 

. . ^ , grammed ^instruction . : . . 

/ ; ' U'C: " ^h-huh- . • ' 

. . SA: ' Now let ' s take a look '^and see if ^ 

there is anything . [ in the ir/zesawrz^s } . 
on types of symbol sy5,tems. 

After' checking through the Thes^ur*us and> a diction'g.ry, the. * ' 

search analyst hits upon the term "ideography.V Other syno- 

nymous terms are sought h,ut very fefw found. She next ttirns 

to •exploring the- concept "computer-assisted^ instruction.'* 




SA: ^ There's going to-be a whole lot of 

- . • terms here, I tjiink. " It's even pas - 
.A — . — — -^ible --^^!^^ 

^ \ insjtead. p£ looking through descriptors 

' only, just saying that the document 
has t<> have the word "computer^*' or 
r-' - ■s^tem.;'* compute rjr soihewhexe.^in> it, - * 
.^i:-'^^'' }^-^ - ■ e-i^the,r ..-in the ^bs tr^C'^'P ^ titl< 

'.. -[^-^i^^:''- ox- ;the ■des-crip.tp^r s?. ; 3|ia;t^;^^ get < ui 

SAi v' .With ^'programmed'! I qan't use the 

: . . stem' ''program." It's too broad* ^I : 

\ ' ^ . would /have to - sax"p^ogrammed" or ' • ';" - - 

' - . .V. - ''programming^:'-'-^ ^. \ - ■ - > j 

. I'll also -use: "teaching madltines" " 
: " y^^^^^^^^^ ' -- that usually indicates soSue.thing T 

close enough^vf-^ M go^ 
. , / so far as to put iii^ t'hings like . i 9;. 

"educational technc^ogy." - - \ * 

U-C: What would that do? ^ . - , . 

- SA:' /■ It would get any kind of gadget or. 

~T— . gimmick", like teaching machines. 

- _ U-g.:' - Oh, really? / ' ^ ' > ; • 

SA: ' I, think so. 

U-C: Well, put it in "because that might 

^ be part of tl;ie computer . 

Tlie analyst draws this -pari: of ; the: interview to a. close by 
assuring the^ user that- she will. examine the Thesaurus for . 
more related terms when she is alone. She now shifts atten- 
tion -to examining the^ concept^of -"brain damaged. " 

SA: , , I ' know "cerebral palsy" is your first 

- choice. But there are :a lot of other 

terms that refer to other kinds of 
' 'brain damage . Maybe we'd better look 

up some c^f those. ; : . 

U-C: Yeah.' :\ ■ - , ' 

* SA: They use ^'perceptually hand.i capped. " 

• . " y-C: With normal kids it's okay too. 

- - ■ /• . SA: That w -i ' 

/^*«^an i^ neces leave, out this 

. ■„ ' . ^- cluster . - : ■ . . . " ■ ■ 

■ There follows an -explanation^ a search strategy will 

- probably be constructed before the analy,st returns to probing 
the meaning 0;f the thir^KCO^icept . ^ ;\ ^ . 



67 



SA: 



1 tbink I was ajSklng you about some ' . 
of the terms related to "brain 
damaged - They -have - -neurologicall^^^^ 
handicapped,'^ f^wd^percep tually handi- 
capped . V ^Thos'ey are too. mild, aren Vt 
they? ■ .^©if^ ar^ " th \ > 

Na;t"nb%-^ia:ily^^^ 

Okay ; I-V 1 i r^ii^ t : them ..down >^ "They may 
get .you * as mahy as- 150 '-'citations . 
W6uld you like to see what form you 
get them in? 

Sure . ' - ; % 



u-e 

SA: 



U-C: 




The :analyst .uses ^sample b ibliographies -to ^demonstrate the. 



.various .fori^a^^ joUrnaJ/^ and dogumerit - citations as well ,af§* 

' '"Te.rent -irjdexes - She ffext. explains the .charging '^srt^ 

;er' s 4jncourag^ decides.Vto s the ERIC, ^^^ .(^ 

Excepti^priaZ Child Education Ab.kzrz^a'C'ts y - a.ri^ 



y^lSi the 



Abstraats dkta. bases: 
SA: 



" SA : 
U-C: 



If I search more than one'T.data' base, . . 
it'll probab ly cost . you more th^ari / " 
$30. Do you think that' s okay *\ 

Yeah, - 'That ' s -okay . ^ 

It could be as much as $50 or $60. 



That ' s okay, 
tt.^ 



I think it's worth 



Though -til e interview, which had already lasted twenty-eight 
miniites, could very well have" concluded here, it goes on for 
another five .minutes" as the analyst recapitulates her search 
strategy and-, having e^ch^us ted the ERIC THesaur^us.'^ turns to y / 
P:Syqho.ZogicaZ Abstr.aats. ' Having dif£erentiate:d between the : 
terms used in the : various indexes , the search analyst terininates 
the interview by informing the user when she may expect her 
search to be run and her off line printouts mailed to her. .Two 
weeks following, the interview, the user ^ received'; a. bibliogfaphy " 
of 1S5 citati^ons, most of which she. felt-were relevant. 



Use2? D 



The fourth request was a relative iy s traight£orWa:^d b^^^^ 
/ submitted hyi an A. student working on her thesis . 
. .. . U-t)': 



ERIC 



I want to do a study to see the . 
effects 'of varipus. televison ^ shows 
on pro- social behaviour and i 'm 
going to use altruism as my -measure.: 

- ■ . ■. ■ J 

•> ■■ ■^ :■ ■ : ■'■ ■ ' "■ ' 
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What I • 11 be doing is showing 
.several dif f er^nt vtypes of shows 45. 
::-t0 ~l:-±.ds^^g gf^s ^ve-:-;SlTdws-; -po si - ~ 
-t ive.-shows and . then have them 
•carr-y^/out an altruistic task and 
see -if the show ha:s affectad themi. 



.1 think; that* kind of thi^g .-Shouldn'-t 
- "be tOQ -hard to. :find beca'us-e.'''there ■:• • - 
are some ver^ specific words. you'r^ 
interes;ted: niaifily in teljCvisidn shows/, 
■ . arp^OM? ' ■ ^ y.. . ^ . 

•:Y-es. 

As^ oppo,sed 'to movies or rkdio? 
,'Yies. ■ -r; . ■; --'^ - - ^ 

• Okay.- Now' 'laggress ion^' or -;"vi61ence'^ 
- -■• • those are qiii'te speci:? icVvtoo . - ' ' 

_ They won ' t be^ too harti ; t.o get at . ' . Oji- 
some topics , Jrou know A" you do so much 
typing and it \takes so\ i^ng that' if • 
yau search two'^data baAdjB^it cbsts ' 
mor-e than , $ 30^^: -But ^''^^^^ thi^ this 

. rone will . ' 'I not «yen- ^^M^tain. ^hat • 
it would_^be 'neGes>ary: to SeW ERIC. 
It sounds more like- Psi/c/i Absi'Taats to 

■ ^® • . ■' 

It' shdlild all be in psyeh journals, 
yes. 

I.'ll just /get out my Psyoh Ahstr^aat^s- 
thesaurusv<iuar^ I tjiink" this is- a. -topic 
where we won* tSjieed a thesaurus too 
muiih. Now., just, while, you're here, 
I'll use ypu,;to jget. all the possible 
synonyms b^e^use you' re' ^he one who ' s 
/familiar vcith the literature. "Aggres- 
sion" and-"violenc'e"^ are two possi- 
bilities. What about things like ' 
'.'anti-social"? . ■ " 



•3 



Yes, because ^one of the shows will^ ' 
be that . ■/ ; ■ ■ ,. . 

••Hps tile, •'^V ''hostility''?.; / / \ 

'Uh-huh, , ;/ ■ \ ■ ' . / / •.; .' 

If . you happen , as we ' re go ing al ong;, i 
tb_ thiitk of a few jnore synonyms , just 
interrupt and I' li write them down. ° 
Now. , we ! II . put down "television'? or ■. 
;!&-adio." - I'll also- pry.t down J^TV" iii 
case, somebody's called' it that^ . ^ 
You're not really interested in . 
movies or- books? - . . .' 



-J 





There follows some' dxicuss ion about the inclusion or exclusion 
_o£ -STTier media'^s^h^ as' £1 lm,--_whether,„or \not _''chilcJren"_^iShould 
be included as a- sepa-r^e descriptor ^^-^ > an <f methods o£ gettii 
at the concept p£ altruisni.': .Defining the latter presents^ the 
^search anai}^st 'wiA:h some difficulty- and' -she ask^s the us^er to 
describe the kind of -situa-^oh to*.which the children wib.l b^ 
sl^ed to Respond. \ ' / * / . . ; - ^ ' - 

- U-.D: What the ^kids have;' to dp i^s bowl oxi ' 

\ , ^ /an el'ecx-ric' p^e^set gai^ 

/ . . ' : ' a certain, num'^er of toTcen and they 

^ axe- given .^he oppor"^\mity 
' . . ^ 4 - * ' ' . ^ "their tokei^^; with the postal: of a 
V hungry la tt Ite • boy^up on 'the wall.. 

The%' are tola tltat i£ they share with 
- ^ Bobby, he's going to^get spnte toys.. 

This elaboration prompts the analyst to suggest' "sharing," 

"altruism,"' "cooperative.," and ."donating" -as likely syndhy- 

mous descriptors . She explains that the use of too mahy'^ 

such terms, however,- might pose unnecessary restrictions^ 

The user at this point reminds the analyst that she. is just 

a.s interested in the^ effect o£ positive shows on "^lildren as 

negative ones. They discuss, :various, approache-s that might 

be taken to get at this information andVthe analyst cautions 

' * **".' '».■,••■■ 

her about being too specific. - , 

. . • ■ . .- • ■ ■ .- \ 

' SA: . Whenever I'm doing something for ^ 

someone- that 'iS "doing a dissertation ./ 
or thesis, I feel obliged to warn ^ , 
them that the very same behaviour 
' _ that you have to -engage in to refine . 

, V and narrow, yoiir thesis topic "^makes 

it increasingly difficult for tlj6 . 
* ^; ^ : , search to come up with anything^ 

^ " ' Almpst all . the stuff we find wi-11 ^' -. 
; ^ \ . be tangential -or- peripheral . Some . . . 

- . . people a:re very disappointed. They 

^ / ^ want thetir topic to be" unique but 

" on the otl^r hand, -".they want to get ' 

' . ' . . ■ a 'lot of things tha,t are'^ similar,. • 

. ' It ' s- kihd:*o.f har^ 'to have! it both 

■ ■• ' ■ ' ways. " - ■ " ■■. .. " / ' ■;- . if 

She concludes by saying that she will probably us-e just the 
TV and altruism cbricepts to start and alter the strategy, after- 
wards, if n^eed be. The user is shown sample bibliography to 
become .familiar, with the format of the printout," and the inter- 
view ends. ^ Two days later , fifty- seven . c^it at ions from the. 
PsyakoZogiaat Abstracts drata.^^base were sent to. the user. . 



Analysis and Discussion o£ . Findings 
; ; Emp arthy^an d.„N 011.^ Ve rb ai_„ Be h a v i our , . „ . 



Earlier, several- conceptuali zations o£ the negotiation 
^^^in,^rview were identified. In this^ sect ion , .these, conceptis . . 
will lie a.|^lied * to the preceding^ interviews to determine their 
•* "closeness o£ fit.'^ Subsequently, in the discussion sect^'ofi, ' 
another view of . the process will be- offered* ^ . . • 

^It is^recognized that the absence of vi-^eqtapes imp^ 
. some limitatiTons on. the ability to fi^riy captur'e the nuances 
of ^facial exp^re^s^£op<"i/the .position of the hands and. body of ^ . '* 1 

■ th(e search ""analyst; ' and to" identify., all exhibited aspects of 
.nonverbal behaviour '=whiclj the search analyst makes us-e .of to - 
prouiot-e . empathy . wi th -'the: use.x>. .The. use Qf tape - r^ecordings^^do-, 
however, make \i.t possible, to- identify 'one category of empa-"^; 
thetic behayibur, . the use ..of verbal and "njbnverbal ' expressions 

> '"said'in'a warm, accept ing and . unders ta.hding jrianner and . ' r r'. 

\ selected t*o fit the context of .;-thA interview'' (Pierce , TL971.); * 
^ \- . A sample of the*'-interviewer ! statements were : V'Uh-hmm," repe- \ 
^ titon *of the laser's last statement^ '^You feel that flas.t state- 
ment said],^' and confirming statements , such - as ^'Yes ,: that seems i , 
to makis sense..// Matarazzo and Wiens (1967), also in the dis-^ 
cussion of the res-ults of their experiment, note that in-t^.- I , 

behaviour such as "Mm-hmm" r , ^ * ' . ^ ; ^ ^ 

^ .1* . ■ - ■ ■ ■ ' - ■ ■ 

■ ■ .. . is int'erpreted by "the interviewee as ' , 

indicating , that the interviewer is more 
interested in, or more' empathi?: toward , the 
^ interviewee ^ . [and] these interviewer T *. \ 

■ ^ tactics may- be. functioning as social rein-. 

. ■ • ' forcing stimuli . • 

. Other forms of empathic behaviour include humour and " 

statements ;_s.uch La "mi-huh,'': •'^Really , 'V and "O^h I 

;V see A further 'implication is that t of behaviour 

■ encourages the .user. :to /reiax^.an^ t^T^ rather than^-^ - ^ 
withdraw into uncomf or-table monotonic responses. Encouraging 

. the user to t-sflk soli-cits information •about his. tSpic of ^ 

instre^?^. whicli is." cruciaX t understanding of his problem* 

Only- i this , first step- is campleted? successfully, can the 
search ajialyst 'mtrtTe^^n to the next- stage of attempting to, 

, interpret the problem - in - terms' o.f^ descriptors on the system. - 
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A particularly good example of this is demons trated^y 
the^ firsts four minutes (six* transcript p'agesj^ofr the" inter- 



vie^ with 'User C -who requires • information on^jhe use of, 
computer- assiWted^ instruction iri teaching symbolic language ^ 



Zo jchl 



Lildren a-ffected by cerebraJ,^ palsy/ ^ the us'er do^s ' 

almos"t all - the talkiTig for the first four minutes, punctuated ^ ^ ■■'^ . 
ohiy^y* th^seafch analyst ' s/"Okay "Uh-Kuh, " '^'FineV'^" I] see ' 
'€he ,sta:^ment "Maybe you coui4 . tell me a little' more about thisA^ / 
symbol, system that being -taught two '^rief . questions 
about the/ type .of fdundati<)n w the- user works and whether" 

she- wants information .dn./riornTal as well as- cerebral palsied / 



children. 



"^In fact, approximately one-third -.of .the^ empathrc com-- 
men.ts Gt>venty- three "'out of a/ total j6"f seventy "^h^^e) -'occur^^ 
during^^the first four minutes-, of "^K-S; thirty-'.three mn'ute > . ' 

interview-' In the inte/rview with l^ser. B, -ia^ .only nine . y 

minutes , . out of ^a . total of-^'.t\venty- eight such comments by .the : \ . ^ 
s'e.arch' analyst^ fourteen, occurred during the first two minutes" 
of the interview,' and-with 'User A almost one-qiiarter of such 
comments- Ctwenty- one out of a total of . ninety} occurred- during 
' the: firSt five minutes; of the forty-minute intervjfew^;: -Iij /.the 
interview with. Us.er D^.althougTi there . ,w^re foVty —siiclr commejits 
in only -twenty- one minute?, only £ive occurred in the first 
few minutes, the reason * being that the search analyst.^iJmedia:tely 
understood the user's atopic and proceeded quickly to- the ne^t 
stage of identifying the ma j or descrig^-feors . ;^ 

As can be seen; most of the search • analyst 's empathic 
comments relative to the total of such comments, uf^ered during 
the entire intervi^^^occur during the .'opening minutes ^*of the 
interview when it i.s vital tl^^t . the\user\)e at ease- and talk 
"about. .|iis topic of interest as fully *as possible . ' This un- 
doubtedly will also influence user >satis f aclfion with, the ser- 
vice^'^^generally his decision as to -whether he will us^ 

the Servifze'/'again and/or favourably recommend it to his coi-- ' 



leagues- and friends. ^ 
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The Structure o^£ Negotiatioil • • ' • . - 

- Even-- the- brief/- descxiptions.^.o£„tlie_.£our^^ 

sented -confirm the underlying five- stage pattern or structure 
that* is suggested by the literature . ' . - Z ' ' ; :^ v ^ ^ 

. ~ Stage One . . '•• - 

The~£irst s t age'/ r^jvo Ives around the . analyst"' s attempt 
.; -to clearly understand the, user • S -request . This seems to be 
. a most" criticaljstage ,^or only aftera its successful comple- 
tion can the search analyst mavei, to the second stage! The 
length: of .' the firsV stage depends on how clearly the user has 
identif ied /.and def ined his topic in . his own mind, the com- 
plexity • or 'Iwiiqueness of. the topic itself {.of. User' A) and ■ 
.V how articulate the users are {.of. Users B and C) in explaining 
' their need to the analyst. The degjree of user articulation - 
l.^ in-this stT^dy tends to be quite hig4C5 . since users are generally 
either -master's, or doctoral* students, professors, government 
administra,tor's , or members of school boards.- • In this stage . . - 
also, us^rs may be-gi-ven- a. gener.al' outline or description (ot? - 
may even .a^k for such) o^ how the system' wdVks. This, tends . : :. 
to provide isome preliminary even if limited \.understa,n4ing-, 
the relationship' be'tween. the' t^rms^ ^^^ how *. 

ac*cess , to- the sysisJii" is obtained j and seems 'a' valuable pre- 
.Ixide in later'- he Jp in g /to identify descriptors-and key-. phrases. 

Stag^ Two • . 

The second stage involves an effort to idejitir^y -the ' 
major areas l.of the user' s topT^of interest and to match, these , 
with major .descriptors. or phrasesrwhicai will prpA^ide'f^c^ 
. to- the data base jCs')y-. -With the first 'St ^eeond • ■ - 

proceeds fairly rapidly . , -Since all the user's are educators 
of one sort- ar' another,- the -terminology tised tends to be ■ \ 
. limited, allowing the s'earch analyst to build up -familiarity 

with the language .of the disci^lin*. - most instances, rele-- ' ^ 
. varit tools such, as the Thesaurus of EBIC^ D:esari.ptors loeated: , 
on th^ analyst's desk; is- consulted by./the :user and analyst 
during theiT .interview. • 
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_ A® il^. _co ine s. t he__l dejiX i.f i c a t i oil . 'b f _..t^^. rms.;. .s.ii- -i;2-i:,€he-. ^ 

"^^joa; '"descriptors. in this stage an attempt is usually made. 

Z^.*^ bro^i^en. and ^xtend as much as poss ible the topic, first . ■ 

. >P.reggnted by the user. In this stage a list of very specific 

^<^rds •i.di?,scriptors) is identified.": This' process ma.y.be a'/' 

- ^iriy l<^*ngthy oae and :w^ith.the poij^sib-le "except.ibn/.'of the. . , 

. V ^i^st^^^g^r, -may take theniust time during the iiitervleW'.. '* 

* . During sta^e .tour^ the. ^ear;.-h analyst outlines .the main 
- ^n.d alternate s ^raj:,e^gi^s <h<)'y^ihl use in the s.earch, la'nd ran.ks 
^^^d/of ^"ombints 'the- re tared t'^vms' u i th the ^maj or. descriptors/ 
^^afviings. This stage usually acts a^ a furc;'her clarification 

th^ ^<^P i c and. the d^pecr^ o r i r • thai are. nros C i/iiportant, 
^nd th^' user 15. ..asked cu fu>thor exccnd, change^/ or modi.fy' ' 
."the-^t^^^'i^ as necessary. ^ / ^ • • 

• iJie closing sta^^is usually de vo't ed ■ to - *'hous.ekeeping 
^^.^orin^^ion'l when- the user's address and billing instructions 
^.^^ ob'^^i^sd and'^he^is tx:ld how much material to expe.ct~-w 
-' "and how much' it ,l.\ke.Jy to cost. ' / ^ 

It sJiourd iid-^'ed tliat ^whiie Stage One' usually occurs^ 
■^^ the .fc*<^gi tilling .of the . infer vie\v situation, rhe remaining^' ' '■. , 
• ;*^^ge;5^'4^ ^ot'^/rieceS'Sari ly f-ol lou. ijequentially in th 

■' P-fes^jjted above . l-"of example y. an indrcat i.on ■ of \vhat the user 
can ^jj-pect in^ terms .of the vo^unie, of ma teri al-, ■ may occu^- '■ ' 

- '.. i^m^-diiite-ly -a-f t.er -.th^'f i fs.t stage i^-/-. User A) o the sear^-h - . *, 

--anaj^y^t may .proceed ro. i'den til y during the first 

, , .'^■'^^'^t^ in an attempt to under stand che t?oi.iiC' more clearly t 
iesp^.(;,.iaily i-f ■ ^ -is V compl'e^ 0'>' uftiHLi^' o^*^J before' making 
«<a^ attc-'iW-: t-o- specify ^and l±U^-maior headings..- In spite "of ' 
. "*''^rrati>;;-"'^~^ ^howeverV basic ^utvdcr-1 ying "pat tern' /abb tends . - 
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OpeiY and Closed Questions 



The ^distvTiburion of. "open" and '^closed" questions asked 
by the search analyst in' the four interviews ' apt»ears below: 



/ 



. TABLE 24 . . \ ' 

OPEN- AND CLOSED QUESTIONS ASKED IN INTER^BtfS . WITH 
^ • . ' USERS A, B, C, AND D 



User 



' Open 
Questions 



Closed 
Questions 



Length of 
Interview 



A 



^ 13 



^43 



40 minutes 



B 



7 
9 



9 , 
27' * 



9 minutes 
33 minutes 



18 



21 minutes . ^ 



In- King's opinion (1972), the .frequent use of the open ques- 
tion encourages the user to talk and therefore ' allows him to . 
shape his request^^ It would be expected, the^refore, that 
more "open" questioi^s would occur in the opening minutes of 
each interview when the user must explain, his r-equest. There 
seems to" be good support for this coni^ntion in the four 
interviews being considered her e In" th)e interview with 
User C mbre-thaji one-half of the open questions (five 'out of ^ 
nine) were asked in the first four minutes of the interview; 
in User D's interview, on^e-half of the total of such questions 
(two but of four) occurred in the first seven minutes; and 
almost one-half of those found in the interview with User B 
(three out of seven) occurred within the first two minutes. 
-The exception was' the interview with User A,oWhere, in the 
fir5t five minutes, only, one out of a* total of- thirteen bpen 
questions appeared. 

The- reason for the lack^ of such qi^stions at the begin- 
ning- of the interview with yser ,A. seems to lie fn the search 
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analyst ' s .attempt to pin the user down to specific^ terms v^ry, 
early in the interview , as , liaving. begun the in1i.^rview with an 
open 'quest ioii , she received the indication that user vs 

topic was an unusual one for which ^there were 'few applicable- 
descriptors on tlhie system. Continuing to describe his topic 
in general' terms , therefore, would be of little use. In. con - 
trast, the lacrk of o.pen Questions on -the whole in * the inter- 
•vxew- with^"U f our) se.ems- to occur -because 

from the user's f irs t explana-tory statement, th$ search analyst 
was ab'ie- to say ''I think that kind of thing wouldn't be too 
hard to finci . . 1 actua lly% ' becau.ic there -are -some very 
specific -words" and began to ask closed ques^tions^ in order to 
obtain a specific list of descfip tots . ' - 

It may be observed that th.e EISO se^ch janaly'st spends 
the greater part of an interv-iei^^ asking about specific terms, 
siiggesting ^"pWticular descriptors from the Thesaia:*us of E-RIC - 

' Desci*iptor*s y explaining to the user what they- mean, and indi- 
cating how ^ she will combine- them in the search strategy. 
Because of ^his need', then, for a lifting of specific headings 
and specific combinations of major "and related - descriptors 
the search anal^ys t ' s 'use of the closed 'questions dominates the 
interview after the opening few minutes.' A brief look at the 
distribution above of the ^ypes of qurestions will indicate the 
following: User A was asked more '"than three time.s as many 
closed as open questions ; User . C, exactly three times as many; 
and User D, more than four times as many closed as open ques- 

, tions for the duration of the i-nterview. Iri^^ bri^ef both the 
open and the closed questions have thei^r- impq^rtance during 
the negotiation process -- the former when general undei^- 
standing of the topic as a whole is required at the beginning 
of the intervxew and when this form of unders t andijig of d. 
"section of the topic is required late?* in the interAriew; the 
lat ter * when the- topic having -been understood, a speai'fia list 
of "majoT and' r^elated desaript^or^s are required* to gain access 
to the system in order to retrieve requested material.^^ 



Role and Status Relationships - " . ' *^ • / . ^ ' 

With regard to the*" <iiues tion of role relationships, the 
.situation in which, the ETSO search analyst -fin^s herself is* 
in many ways different- from that' of a reference librarian. 
While the latter may find the interviews unscheduled and open- 
ended, the search analyst oft'en makes ^an appointment with^, tlje 
user-beforehand, and many times has at least 3ome.- prior know- 
ledge of t'he user's topic. Furthe.r,^ whil,e^ lack of timV and 
the competing attention of other users may be yariab les^N4n ' 
the library situation., prior appointment atid-the fact 'th^t ^' 
th.e EI SO o.ffice is ^cated in a separate room "in the library 
of The Ontario Institute for Studied, i^- 'E^^cation, effecJ^dKely 
eJp-minate these difficulties. - y 

In fact, the difficulties mentioned in the review of 
the literature on the 1 ibrary/library- user in^terface, the ^ 
disparate functions of refe'r^nce C^hich is not a quiet^acti^ 
vity) that annoy the quiet- reader , the distance --be tween th^' . 
desk and the reference collection the lack of objective 
measures to judge effective service, all of which affect. the 
successful.^ establishment of status and role relationships, 
are not present in ' the search analyst/user situation. The 
closed office ensures the search analys't's undivided atten- 
tioh; the "reference section'* comprises 'the Thesaw^us of 
ERIC Besar^i.'p^tOTs arid other indexes that are always on the 
desk at whigh the interview takes place; and ^he search 
analyst is provided with "some feedback on the efficiency and 
standard of .her performance by means ^f evaluation question- 
naires r^:^-'rned by users. ^ - i 

Iniri'aiay, also, the se-x of the user was presumed to 

be a variable in both status and deference patterns iix the 

interaction.. However, bearing in mind the limitations of a 

tape-recordirig.., this" factor- was not' found to affect these 

patterns. Thi's* finding has received further support, from ^ 
^ *"***, » >^ 

the results, of a separate and indep'endent inquiry of anothe 

researcher ^who conducted- follow-up interviews with users of 

the EISO service wiiich included the user group from whic^ 

the . four considered in this aij^alysis were selected. 
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rtowever, there can be one factor, which nia^. have an effect 
on'. raising the status ofVhe sea rch analyst vis^^a-vis the user, 
which inay_.result -in a feeling of dependency 'on the .part of the 
. user with' -added discomfort to the user^. especially xf he con- • 
sideris himself to be in a "high status" occupation. . That is 
the perceived glamour associated with a new and sophisticated 
technology. represented by the pomputer and the possible per- 
ception that "esoteric knowledge^^is required by the search _ 
analyst to retrieve necessary mate^i^als from such- a system"!. 

Yet^ there seem tq- be several factors which .-jjreclude 
this possibility, ^ First, the fact of a prior apjfDintmfent 
seems to set a professional' tone to the interview. Second,^ 
while the reference i ibrar ian/user inter^ctio^ is free of 
charge, this does not apply to trhe - EISO*" seryicfe . The user 
would therefore tenjd^ not to regard the search analyst as " ^ 
doing hiin a favour, but as entering into, a business transac- 
tion with him, and, as such giving him the right to expect/ 
.certain acceptable standards of servi-ce- Third, it Was coh^ 
.sistently 'found that the search analyst, during the interview, 

not" .only atte^npted to explain how the system- worked {in fact 

■■ - ^- It ■ - ' 

users ^ theiijselves often ask about this), "but/also discussed with 
the user major and alternate stYategdes that she. would use in 
t^rms of^thj^ system y to attempt to retrieve , what the user re-i 
quested. : Thi^^Stended towards a reduction of the fee-lii^'g, in 
the. user tif -it existe'd'at all) of Yhe power of ' the search 
analyst's "esoteric knowledge." It /is worth noting that* the 
majotit5^ o£ .new EISO users (85 % from previous research) report 
re.arning something ai)out, the EISO system itself after the 
interview ."^^ ./The four users in tljds study who had completed 
and re tufned^their User Evaluation Questionnaires reported 
that they ha'd learned <"a great deal" about the EISO: system 
and- how it worJcg. " ^ ' ^ ■- ' 

While'there was no role difference, in clients of dif- 
fexent sexes.^^ there was a difference *that appeared'^between 
academic *and -nonr academic clients. The ^ two academic clients, 
both of whoi^^ were by chance master's" students in psychology, 
stat.ed their questions in terms of formal, research designsi. 
The first wanted to control for socio-economic conditions by - 



having only children o£ lo^status, but had identified both 
experimental and control g/roups . Members ' of . the first were 



to have low self- cojicep"^ 



f ability and the second high self/ 



concept; the dependent vai-iable was perceived social • distance 
from groups of prof essionals . * 

\ The second master's student also had ident i f ied ' an ex- 

pe^rimental variable type^ of TV shows and a dependent 

y ar iAb 1 e - - 2^1 txuism__ as a _mfe a's ur e^ o^ p r o - s o c i a-l -b eha vi our . 

In contrast, ^ the qup^tions of the two non-academics 
Were ^nore open-ended. ^One wished to know about' the role 
school librarians and children' s reading habits ; the other ^ 
about computer- ass is texi instru-ction for teaohring brain- 
damaged children a mo.de of Symbolic communicatipn • . • 

' • t It feeling .that\the ra-ther rigid' d of 

the problem statement 'in the interview with User A may .have* 
cojitributed to ^he large number of questions and failure of' 
the bibliography to provide 'highly relevant mater ial X^iven 

— that -the search analyst wasHiot familiar with the line"- of ' 
research ''in which the . client was engaged. Even at the end 
of the Interview, it is questionable whether this wa§ clear 
to her. In contrast, although the topic in. the^ intervievs^ 

.with U^er D was stated in almost as. rigid experimental terms 
>Jnas'the .fi-rst, ^:he search analyst 'immediately understood the ■ 

"^topic, and conducted a brief and successful search. - ^ ' 

. / This cojtt'i'ast suggests that academic .ushers- may .tend to ■ 

statei q;uestions in a relatively formali2;ea manner" that ' 

- ;^makes inquiry on the part of the search, analyst* difficult ' if 
^ -'she is not . equally familiar with the -paradigm, and concepts 

«iinderlying the -quest ion. At the, ^ame tjjne, the -.can-vent ions 

♦ -Opf the- problem ^-atement.-fiTay. tend to obscure what is in fact 

a poorly- defined question on the part .*3b£ the user,:, -.giving trie 

• -."i^ues-ti^n fh^i^afjpearance of a validi-ty.-^hat- it in fact lacks. ^ 

Non-*'academic u^ers have no such Titasque .for the^r confusion; 

thed'r ques'^t^ion is still at tHe ylscer^al rather than the . ' 
cognitive .stage*; it will be evident'. * - ' ' . . 
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^, . Developing a Preliminary Model o£ 
- " the Negotiation Process ^ V ' * ■ 

While pecogn^'i z ing the clanger of trying to generalize . 
outward from a small sample of interviews, a number of points 
^nevertheless are sug^gestive. First, that sojne of the issues * 
raised in the 1 iterature' ' on the tradit ional reference inter- 
view are also helpful in studying online negotiation processes 
The aspects. . that hold most relevance seem to b(5 those asso- 
ciated v3*9rth empathy and non-verbal -behaviour, thfe structure' 
of the i^gotiation, and the use .o'f open and closed questions* 
The one aspect that does not seem to. b-ear- much relevance is. . 
that related to role and status relationships* Second, these 
Elements ^em^to be constant at least in :some degree regard^ 
less of th'fe -nature' of the ..topic , rhe user, or the system used*. 
5uch an observation leads one ^ to the notioji that , it may be 
possible to represent - these stable' element s ^ by constructing . 

a model. Such a preliminary attempt 'to visual'iz.e , the nego-. 

* 

.tiation p'rocess Is presented in Figurje 2. : 

From the' .analysis of the ' i^iteryi ews it became '^clear 
that the negotiation process is- hot a rigx-d progr-ession with 
a clearly de'fined beginning, m^iddle, and end. that inexorably 
follow upon each.o'tne;r l^tit rather it has a more iterati\7'e, 
rep.etitive 'nature as ^the. determination of the specif ic require 
ment o'.f the user emerges. It therefore could not, be ^represen- 
ted by a -simple Linear- design but called for a more dynamic 
mode representation. The ridtions of reiteration and 
dynaniism brought to mind the model of the spiral 'curriculum 
in*^duGation and the doubly heLix in jnolecular biology. The • 
former incorporates the idea of returniixg ^to concepts already 
'introdu<:ed to define and learn about them'in greater- detail 
-- ".a process the analyst uses as s-he moves from a .general 
understanding of the user^s topic to the more specifd^c choice 
o^r descriptors and search strategies while the latter draws 
atteorifion to the multiplicity of interwoven links that are^ 
b6"tTi' constant and different in 'all of nature. Obviously, 
close parallels cannot be drawTi, but the no.tioji of using 
spiralling strands seemed to combine continuously evolving 
specificity with dynamic interaction. Unlike the ej-egant 
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simplicity- of the double helix, 'however , the model of the 
negotiation process was pictured as having at lea'st four com- 
pojient elements. These are associated with the yser, the 
ge.arch * analyst , the topic to be searched, and. the system to 
be used for searching. It is not; the representation of these 
holistic elements that is the . ccyLfcern here but the depiction 
of the elements that surface as being important during the 
negotiation process. ^At a fhicro level these have beten iden- 
- 1:1 f i e d a s emp a'l: hy an d~ ti on - ve r b a 1 b e h a vi o u r , ;^^t r ucTur e ,'7and "~ 
open and closed* questions • At a. more macro r^eJ,. *the impor- 
tance of a specific characteristic such as empathy can be 
broadened to stand for behavioural charjg^cter is tics in general 
and the specific method' of using op^n and closed questions 
for a broader range of techniq-ue*s us^d by the search analyst. 
The notion of structure is incorporated by the sequential 
progression" of^ the interview -in stages through time. In 
addition, there are at l^st two other strands that weave 
through the typical negotiation process, the substantive con- 
tent of th^ request and the technicalities pertaining to the 
choice of descriptors , the -formulation of search s t rategies 
and the operation of the sys«^em and data bi^se chosen. The 
■first' of these .is re^p resented by the term '\topic," £he second 
by '"'system* " ' • ^ • 

Whiie ^ the mode 1 is, meant to be suggestive of general 
phenomena, it could be adapted to more closely represen>t 
speci'fic negotiat^ion interviews. For e^ampLe., ,,.3.tratids may 
be broadened or narrowed, lengthened or shortened to show 
when and the extent to which individual elements occur. Th^ey 
may be divided and color-coded to emphasize the prime parti- 
c^ipant^ in the process at any 'given time, whether* it is the 

search-^ analyst or the user. Specific environmental influences 

/ 

may be shown to effect the turns the interview takes. Such^^ 
detailed mapping, however, must await in-depth analysis 
derived fro(ri^ a larger sample^^f ne^gotiation interV5.ews. ^ 
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Limitations 

The conduct of this study was subject to a* number o£ 
limitations that severely restrict its generalizab^ility . 
Since data were being drawn from an existing search service, 
interviews could be taped only o^" those users who voluntaixily 
submitted requests during a. given period. Thus' tye type] o'^fX 
user was restricted to the kind o% clients using EISO. ^is 
sample was further reduced by the need to rely on the user to. 
return filled out evaluation*, questionnaires to ; complete re^ 
search files* . ^'x^ 

In addition, because real users of a .specific service 
were being studied, the nature of the questxons being searched 
could not be subjected to control. In this regard, it may be 
argued, however^, that the type of topic being searched^ was at 
.least partly controlled by the choice of the academic, non- 
academic ' dichotomy . Academic search topics were expected .to 
be broader and more abstract than those for non-academics 
which were expected to^ deal • more with immediate applications,. 
It will be recalled also that only four cases of new users 
being interviewed by a single search analyst are examined. 
It remains for another study to use a larger sample, comparing 
new and repea-t users with multiple search analysts searching 
a -^ariety of sub j ect. areas . Finally, it is hoped that further 
sti^tdies in this area wj.ll use videotapes of the negotiation 
interview csince so le^4el4-ance on tapes and transcripts do not 
reflect the extent of 'lise or importaifce of body language. 

^ , Conclusion " • 



It. is evident that the warm and friendly personality of 
the present E-ISO" search analyst is a" major influence ^n us^rs ' 
feelings^ of high satisfaction with regard to hel:. helpfu^-ness 
and the interview process. Some other f actors , ^-however , do. 
emerge from the preceding brief and exp'lorator/ analysis. of 
the four user interviews. \ ' ^ . ; 

The succe^ful c-ompletion of the fir^^t stage, .of the 
interview process, during which an understanding of the user's 
problem is obtained seemed to be crucial tc^ the-^ success of 



the entire interview. _ The use 6f the open question, questions 
pertaining to the organization for which the user warks' and 
the reason for -the search/ all seem to plky an important role 
in "providing "the search analyst with the injf ormat ion neces- 
sary for the success of the crucial first -stage of the nego- 
tiation process. Hoi^-ever , . it is even -more important that 
•these questions, whether open or closed; be asked in a warm,, 
accepting, and friendly mknner, since the f-C^st few minutes of the , 
--L-ni^er^oi.^^- have an -irrTpor>tayvt -hecxrliia- 'on -i ts- progz'ess " and' evertiyuaU ovLtcome:" 
If we agree with Taylor (1967) who ,- in his' review of studies 
done on -the interview process, found 'that ease of access to 
information, was more important than the quality of that 
information, then it is essential that the search analyst 
di-splay -empathy towai^ the user. Such comments as "Maybe if ^ 
you described this in a little more detail," "f have the idea 
now," ^-Uh-huh," "Hm-mmm," "Okay," all ' seem to" aid in this pro^ 

The deve*lop7l^^t of both a major list And a related list 
of headings in the presence of and with the help of 'the user; o'ut lining 
the stratesv that will be used to combine these terms in order 
to retrieve the desired material from the system; developing 
alternate strategies, and suggesting possible extensions to 
the topic; all seem to make the user feel more confident, as \ 
he understands more and more not only about his topic, but 
about the process of retrieval from an dnline computed system 

* the start pf the 



which he knew little or nothing about at* the start ^f the 
interview.. 



1 

The user is also entitled to a clear statement of 
:-ealistic expectations. As soon as possible after the inter- 
-^iew begins, and certainly before the computer search is . ^ 
executed, he should be apprised of the possible dbs-t of the 
search,, especially if the search analyst believes it will 
ex.ceed the standard fee. This is even more impo'rtan^ wher 
the cost of a search is a flexible matter. He Should also 
be informed how .unusual .his topic is with regard to the des- 
criptors in the system, and given clear indicaitions of how 
relevant Cor tangential) the material retrieved is likely to 
be. ' 
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Our own .exi^tience with users indicates that even though 
low satisfaction may be expressed regarding the v^lue of the 
retrieved- material, high satisfaction with the helpf ulnes s ' of 
the search analyst is almost always shown. In..^act, those 
users who have been disappointed with retrieved items have 
almost always placed the reason for their disappointment on 
the fact' that the data-base itself did not contain the' 
desired material.. And, the majority reported that, in spite 
Pjf _ t hi s., „.no.t „on.ly wo uld th ey us e t h e • s e-r vi ce a ga in-, -b hey - 
would recommend its use to colleagues and-friends. —-l^s seems 
to point. to the inescapable conclus ion\ that , regardless of 
^the structure of the interview, the types of questions ksd, 
or evea the f-inal. documents retrieved, the ultimate success 
of an online' retrieval service depends to a great extent 
upon an empathetip analyst. Such a cohclusioji has obvious 
implications for the - lection, training, and evajrual^on of 
search analysts;. v • 



CHAPTER 5 

V ' SYSTEMS EVALUATION' OF UStR SATISFACTION WITH EISO 



. ^- ^^^^ 

AccoTTipanying the widespread introduction of online bibliogra- 
phic retrieval services has been a recognition o£ the xieed for 
their careful evaluation. Several approaches have beeir taken 

to this problem including price-demand analyses (Cooper and 

• * 

DeWath, 1977), organi z*at ional studies C^^x, 19^76), data-base 
evaluations (Lancaster^ 1969), and end-use^^^s (BVickley ami 
Trohoski, 1974) . However, becaus^e these stu-dies have taken 
relatively narrow, perspectives, they^have failed .to' assess 
the inter- re lat ionships of various compon(snts of the services 
in question. In order to facilitate this type ojE analysis, 
a comprehensive framework for the evaluation of online^ biblio- 
graphic retrieval services is needed. The purposes of this 
chapter, a-re to present one such framework, and ;jto provide an^ 
example-of its use. 

A systems approach was used in developing the framework, 
incorporating th^tradi t ional categorie's of input, process^ 
output, and feedback- In its. application, particular emphasis 
is placed upon assessing the relationships between variables 
of interest ^at different stages, such as input and o,utput, . 
while controlling for tJie effects of intervening variables 
It is hoped that in achieving a better understanding of such'^ 
relationships, those . operating search services can make more ^ - 
rational decisions about policies and p.rocedures 

- . . • - 

V Systems Framework f of . Evalua tioh 

In the evaluation framework proposed here, a systems 
approach is used to describe the- Cycle of activities that ' 
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occur* -whenever- an > online bib'l-iographic retrieval service . - 
responds- to. a useri^s/ reqifest/ In doing so, fo'&r b4;Sic ,qfues- '^^^ 
tions'iaus't be answered. ^ What iilputs^ «are requiVed? IVhat 
-processes take place? What b.utputs are* prodijjceo^ What feeCd- . 
b4ck ^occurs at '^eacli'/ stage? A description' of the \seq\tence o£ ^ 
evekta- that occur during' a se.arch cycle pravides answers t6 / 

. *- • • ' ^ ^ 4. . C- . - - 

the'se^ qti.es tions^ • - ..^^ * V . . v ^ : v <• " 

• >/ • TJi9 seax^.' cycle -begins wit^5th^ submission a'^-rfequeSt. 

- "t" ^ ^" • - ■* ■ " ' ^ ' ' ■ 

by. a/us>^r to, the seanch. analyst , a' specij^^lly trained i*e£ererice 

^ib.rariaii.^' ' They .theri.;, "negotiate'' ^the^ que*5t^pn, - iin^tTil it can 

be xestated in a f ormf tlrart the' seai:eh ana$5^t . c^n ^^ploy ^ to 

' develop a seVrch strategy that can' be used-^-^i^fe^^retr i^ve 2^1e~ * 

van^\ .citations' from appropriate, da ta-ba^ ^as ^ti;ieyEiiuca- . 

tdonal Resources Inform^ttion Center (E^lIC)* or the Ontario 

Educational Research ^ Information Sys^em'. (ONTERIS) files. The 

.analyst then queries' t^theld^-ta-base (s) ■ using. a retrie.val system 

such as JLockheed Information System OLISO | ^ System , . 

Devel6pment/Corporation ' C'SI)C)>s ORBIT, or United Nations Edu- , 

cational, Scientific arid' Cultural Organization (UNESCO) ' s 

CDS/ISIS. When, combinations of. terms are entere^d, the number 

-of. relevant citations and a few sample citations are printed. 

If th-e citations are judged to be re^levant to the client's ^ 

question., the complete b ibliography ^is printed offline and 

mailed to, the requestor. 

' The^user, on receipt of the bibliography , reviews its 
contents and makes a personal assessment of its relevants and 
thoroughness. As. a result, a request for original /documents 
may be made.' When these are delivered and read, the search 
tycle , as we_define it, is complete. - . 

' F-eedback may occur at almost any "^^stage of the search 
cycle. During quest ion. ..negotiation the search analyst makes 
inquxrie^* and suggestions about the search topic, and the 
user r,esporids. During *the retrieval of ref erences , ' a dis- 
covery* tfeat. an* excessive number of citations have been located, 
that citations are not relevant^ to the topic> or that other 
terms. should be-u^ed^ may : cause -negotiations to be reopened 
and the- search strateg';;^ altered. Even at the end of the entire 
cycle, "feedback occurs ^^^hen an "iKT&s^med and satisfied/ (or unin- 
formed and dissatisfied) user returns an -evaluation questionnaire 



This ^cycle of activities forms, the system- por-tirayed in 

'Figuz:^ each stage, three "actarsV are Involved: the"^ . 

user, the search alvalyst, and the rert'rieval^ system, ^ Each of 

^ the: three • is Jthe referent for a number of va-riables Which, 

describe the characteristics of ■ the search service 's inputs , 

the nature of its processes, and qu^-liti'es of its' outputs.' 

. This sytems framework .provides, a temporal sequence . to 

^ '"^ ^ : ". ^ - ■ 

events/ .© ccurxing during a search Qycle, tTiereby al'iowing " the 

" - * * ' / ^ . y 

/evaluator to make reasonab'le inferences about cause andf ' « 

^ffeet relationships. -This is important, since policy* maker 

must have a realistic idea, o^ the ultim'ate effects of -dif-; 

feretnt -decis^ions about inputs and^the organization of proces 

sing, activities. In"^ a^ddition , the -framework suggests a cate- 

gorization schema for variables, a'ccording to their roles as 

measures of inputs, processes, or outputs. However/ it does 

not provide a guidanc.e in the selection of variables to be • 

nieasured. ^Por- this-, ^we turn to the s^ubstantive disciplines 

of ^sociology, psychology, and economics. - - 

Disciplinary Perspectives . 

The various academic disciplines offer the evaluator a 
large number of 'subs tantiv.e theories and concepts that might 
profitably be employed in the selection of, variabl'es to be 
measured and, subsequently, relationships * to be •analyzed. 
Those concepts that we felt were of particular value in 
evaluating a retrieval service were those of role and status 
from sociology; knowledge and opinion from psychology; and 
cost, price, and demand from economics. The first two in the 
list provide direction in collecting data on a service's 
usejrs; the second two- "suggest how one' might determine the 
'effects of a^ service on the individual; ar^d the third *^group 
identifies factors^ related to a service's fiscal status. Cer 
tainly other concepts could b^ selected as well, but' these 
seven seemed to be ad^qiiate since most of the variables ' 
reported in the ^ literature on search- service evaluation could 
be considered as measures' of one these concepts. 

The concept of role as it iV^used in sociology is not 
-unlike the concept of role in drama. The actor, or person, 

^ ' ^ * ' 8 8 - . - • 



./ * 



J 



:FIG0RE 3 



* . •* INFOPMATION SYSTEM MOIffiL 



Input 



User 

Analyst 

System 





♦ 

Process . 




Question 
"negotiation 


'm 

Reference ^ Document 
retrieval ' retrieval 


Xnowledge 
^acquisition 





... 



ERIC 



r/0 



has a part to play and is expecte^i, te behava, in certain ways, 
^ and not others, both. by hijnself and those aroiind him, i.-'^., 

his role set. A person can expect to have many' different roles 
at different, times ,^ but -typically plays ^only one rol-e at a 
/'^^ time 'COwens, ^^.0^ PP- 74-77)* , . : , / 

V ^ . By understanding the various^ roles played by a search 

; service's clients^, one can understand-, and' even predict user 
behaviour^' and reactions. ' Since" requesting a literature search 
.is an act primarily associated with institutional as opposed 
• Ap pri-vate life, it is apparent that the individual's pirofes- 
• Sciong.1, roles are of primary interest . ' \ 

i - Social s^tatus, the second .sociological concept used to 

^ , guide the selection of variables, refers to a person's s^tand- 
ing in/ the community and the deference the person can expect. 
' * It -has *^been 'identified as ^n.important variable in the li-^era- 
ture on the^ work of . refereAce- ^librarians since the reference 
inte'rview often reverses^ status positions. ^ In^a particular 
instance, a user who holds a 'position of high status may^ be 
forced to admit to being ignprant, thereby accepting a tempo-^ 
rary status inferior to that- of the reference librarian Such 
a humbling experience may not be^sily accepted by senior * 
professors or administrators and their personal reactions 
might create a barrier to the open communication of . th.eir 
information (Shosid, 1974). Ultimately, this could affect ^ 
their level of satisfaction with the service. For thiis 
reason, knowledge of the social .status of a servicers liters 
is of yalue . ' - . - 

The nature of knowledge and the process of learning 
provide the subject matter 'for learning psychologists' who 
have developed' a number of theories that might be useful in 
explaining why an individual requests ^information and how he 
^6r she learns from it. These theories tend to ^agree on' three 
basic precepts- First, a learner must be motivated to, learn. 
This motivation, may be based in the individual ' s personality 
(which opens up another 'realm of psychology^) or it may be ' 
present in the individual ' s : environpient , such as the role 
the person plays. • • 
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- Se|:ond, a learner must possess some prior knowledge in 
order to learn a new concept or develop a new understandiijg, 

: A person almost totally laeking in knowledge about a tqpi,c I 
would probably submit^ an inadequate search request, question 
negotiation might "^rove arduous, and the end results-might 
prove Less .than satisfactory.. / ' ' . ' 

. Finally,, regardless o£ whether, a .simple fact or a com^ 
plex. concept involving gene ra-13?^: at ions and inductive leaps is 
learned, .the *rearner is changed in t^he process. . ^ - ' ' 

* These three generali'zatiorts about the learning process ^ • ' 
emphasize the '. importance p£ kncw^dng the extent o£ a user^s 
mo^tivation,' prior knowledge, and new knowledge .gained as a 
result of a given search. ' s.. 

. As a-^ search is conducted 'and the user reviews the results, 
another psychological process ' comes into play: the development 
of opinions. Opinions , which^ reveal a predisposition to act in 
a specific way toward some object (the so-called^ ^'attitude ' 
object''), constitute a major -set of output " variables . Pre^m- 
ably, opini\)ns about the quality of a search service arid its . ^ 

. products determine ^the ' likelihood of a user*s returning to 
submit additional search requests>v Indeed, the recommendation 
that user satisfaction be assessed on a regular basis is per- 
haps^ the most common recommendation in the literature on ' 
search- service -evaluation. \ ' ' 

The concepts 'of cost , price, and demand,, drawn from the 
field of economic-s , also guide the* selection of specific 
variables. Costs kre typically broken jdown into fixed costs 
(or Vverhead) - which represent the dollar^ expe^nse that con- 
tinues even when nothing is, produced,- and variable costs, that 
are incurred when reduction is raised to a given level. For 
a/search service, fixed costs include salaries , equipment 
leases, etc., while variable costs are primarily associated 
with the charges made by companies providing^ the retrieval 
se.rvice for connect- time , citations printed, and communica- ' 
tions. Costs may be -presented as total (aggregate) costs, '^or 
in %erms of unit costs. ' . - " ' 

/ , Setting prices has been a cx)ntroversial and^dif f icul t 

^^mat.ter for search service^, first because library services 
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traditionally have been- pr-ovi'ded without charge tftt second 
because- of poor data on-total" costs . ' A related concern ^is th 
r-elationshi'p- between price a.nd demanji* . Without knowledge" o£ 

' th> elasticity jof, demand (i .e ., ^the' relative change ip demand 
that occurs in jresponse to a change, in prrce) for searches ,• i". 

' is dif ficult td^ te-l:l what,^i£ any., 'effect different pricing . 
policies may hkve. ' Hence," measurement of price and demand' 
are o'f cri-tical importance in the evaluation of search serj-^ 
vxcesv .• , 

vA wlde array o*f- variables must be me-asured to" provide 
data necessary for various types ' of - economic analyses . ' Some 1. 
such„as staff sal-aries, are input variables which would "be A 
used to determine fixed costs. Others, such as the computer' 
. connect-time for a given_ search', are process variables which > 
would be used to compute^hrariabie £osts and, perhaps," cust/JHte-r 
prices. These data may be-'used in their raw form for soi ' ^ 
analyses or aggfv^gated into totals for others. 

. In the preceeding description about the way in-'wliich 
concepts-drawn from . the disciplines of sociology,' psycl^logy, 
and economies can b.e used to, guide the selection of varia^ 
4:o ba^measured In evaluating a search service, little was said 
concerning the level or type of analyses that could be con- 
ducted. Tlafxs ommission was purposeful, since it is anticipated 
that the framework can be applied at different levels aiid to 
explore different relationships.. In the application that fol- 
lows, ^data are treated in disaggregate form; in contrast, an 
analysis of price-'deraand woul^ require aggregate data. Also, 
the application is "-iriter-disciplinary" in that relationships 
between several' sociological/ psychological^ and economic ■ 
variables are investigated, though a discipline -based theory 
could just have easily been tested. . In short the level at 
which the framework is applied and the patticular analyses 
carried out depend upon the specific questions the evaluator • 
or researcher is trying to answer. ' ; • 



.* . - Application o£ Model- / 

" \ The isfem'aifider of- th'is chaprer- "describes the ' evaluation 
Of two aspects of the^. E'duca't idnaT Infierniation System for ^ 



• Ontario usin|; the systeihs framework portrayed in .FigurerT'S. 

• *iThe purpose of the^ first application is to dete'rmine to -what 
er^tei^t; the client ' ^ personal . and prbfesis-ional background is 
.T®-^^'^^A "to 'op^;|E^r sat is £acti with EISO. "^he purpose o*f 
the second is to ; determine<- the-'relati-onship among the pro.ce- . 
dures- ;fol]h5wed^in processing a search reque^-t and the usef'^s 

. gain in kno>rledge an^ degree o£ s-atisicaation. :r Together , ; these 
assessments should provide a "good understanding o£ the factors*' 
affecting- user satisfaction with the service and thereby sug- 

^ gest methods 'of maintaining' and 'increasing it. • S;^ 

■ ^ . ■ A . ■ ' k ■■ ' ■ - • --^ ■ 

-y •- ^ .Sample ^ " . 

TheV population for this study is the universe 6f all 371 
searches requested from EISO betwe,en May 1, 1976 and April 30, 
^ 1977. Thoi^gh some data were available on all searches, the 
. present analysis was restricted to those which had been run 
on ERIC and for which ^ithe requestor had returned an evalua- 
tion questionijaire. The. total number was 148 (40% of all 
searches) , but missing responses on some v^ariables result in 
fL smaller number being used for some aii'alyses. 

Virtually all searches were placed by uSers from the 
. professional education community in Ontario, which EISO was' 
designed to serve. In general, 40% of EISg*s users were from 
: sch-ool boards, 40% were from OISE, 10% from other universities 
afid 'faculties- of education, and 10% miscellaneous. Appro3^ ' 
mately 30% of the users r^eported that they- were engaged , in 
' administration, T.7% in teaching, 12% in research, 18% in 
graduate study, "7%' in librarianship ,#and 14% other. ' 

Methodology 

Path analys'is was selected to analyze the rela'feionships 
among input, process, and output variables/Tbe'cause - it provides 
. a method- .of tesjting the validity, of causal inferences for pairs 

•of. variables while controlling for the effects of other vari- 
• abl-els. (Nxe",/ 1975, , pp. 383-397) . In addition, path diagrams 



- provide heuristic "portrayals of systems relationships which 
are well suited to-the systetis framework used to organize 
the variables in thrs study. *- • " ; .' 

The analytic technique 'used in this- investigation "fol- 
^ lows the pract-ices advocated by jGoldberger '(^1970). - In particu-. 
^ lar, t]^e:.path coefficients presented here are the stand/rdizVd 
regression coef :|icients resulting from' regression of the 
^^dependent variable, in qj:ies6i on on "those^ variables liirectly 

affecting it, and those alone. In cases where thSre ±3 but 
. . one predecessor variable , the pa^h- coefficient is i(^entical 
to the zero-order correlation coefficient; in otherl cases' it 
-is equivalent to the partial correlation coefficient of- the 
dependent variable on the, standardized predecessor variable 
controlling for other v^ri^ables directly affecting the ,depen-/ 
dent variable. * ' , ^ ^ 

• The 'amouint of unexplained variance in any dependen^t 
v.ariable is assumed, to be accounted for hj' hypo the ticaT) 

- "terror" variables de noted by ei-. The corr/lation between 'e.i 
-:and. Xi is equal to /l - i?^ where^R^ is the multiple correla- 
tion between Xi and alj predecessor variables directly affect- 

'ing i t . 

Not all relationships^ fors^hich no causal inferences ar^ 
plausible may be omitted/ ^ecoi^d, relationships between exo- 
genous variables (portrayed at "^the le^ft of the path diagram) 
are typically unanalyzed, tl^ough they are sometimes controlled 
for if they are considered to interact with one another. 
Finally, correLat ions among the e^ are assumed to "be zero and - 
remain uHanalyzed. | 

^ Path analysis requires three principal assumptions . which 

may or may not be valid.. First^ all relationships are assumed 
to'be linear; "second, relationships are assumed to be additive; 
and third, it is assumed no interaction Ci • e . , multiplica-feiji^e 
effects) exists among ;variables . As well, some authors would 
hold that the variabiles are assumed^^trS^be measured ori a ratio, 
scale with some . fixed unit of me aspire * 

For the dichotomous variables^ such as sex, and th 
measured in Scftandard units of measure, such as the turn- 
time, none of the *assumpt ions would b.e viol^ated.- Those 
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.-'t, ^ Likert scales, such as satisf a^tionj^may violate assumptions 
as to linearity and unitary measurements -However, because ^' 
monotonici if not linear, ."relations are expected for^'all re- ' 
lationships involving these variables, violation of ^the first 
assumption would, at worst ,^ clause -the relationships to be ' ' 
undere-stimated. * . > ' 
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Collection of IJata 



.=.. > ^ The 'major sources of data use^ in these evaluation^ were 
■a Use^4,Evaluatron Questionnaire (Appendix A) and a Data Sheet ' 
'^^^^comp^te'd by the search analyst (Appendix^ C) . In all, 131 dif- 
*f^^nt variables we reV» regularly collected for each search that 
ya.s completed.' Selection of thfese items was- guided by the 
discipline- has ^d concepts noted earlier. Most were drawA/ 
from the literature of previous evaluation studies but only ' 
those t|iat Kere/defens-ible jneasur-es of roles,, status, knowledge, 
opinion, cos t price , or demand were included. Where gaps ^ 
appeared, additional items were ^e-velo^ed, - ^ 



Measurement of Variables 



I, Descriptions of the items used to measure the variable^ 
included' in. the first analysis which focuses .on the influence 
of the sociological backgrou'hd of clients -on their level of - 
satisfaction are listed in Table 25; those for the second 
analysis which focuses on-t.he influence of process variables 
on client satisfaction appear in TabLfe 26 . - . . / 

The first five variables in Table 25 are input variables 
related to. the -backgrounds o_f - clients. All are dichotomies 




zation in -which the user was employed or enrolled as a sZud^nz , 
■SIX needs no explanationf^ ORGANIZN is (5ie organization in 
w^ich the user was employed or enrolled. as a student. The ' 
PURPOSE of the search refers to the use made of the search, 
.whether it was for professional or academic purposes. 
PROFROLE indicates- whether the user was- in an administrative 
or non-administrative professional pos'ition. - , 



^ ' > TABLE 25 

VARIABLES USED IN EVALUATION OF^USER'S BACKGR0UM3 
ANP LEVEL OF SATISFACTION 



Variable*^ 



■ Item Description and Coding 



Number 



'Name 



Description 



Coding 



S ,4 



7 

9 

• 10 
11 
12 



LOCATIC^ 



SEX 



ORGANIZJJ 



PURPOSE 



PROFROLE 



SYSTEM 



SA.TIS6 
SATISl 

, a\TIS2 

S^TIS4 
SATIS5 



5 



1 

j 



Metropolitan Toronto . 
Out side, Metro TorotitD-, 

Female • 
Male ' 

Student or staff of OISE 
Non-OISE, 

Purptose of search: 
Non- academic 
Academic 

Professional role: 
Administrative 
Non- admini s trat ive 

Search system used:' 
SDC ORBIT 
LIS DIALOG 

Overall satisfaction 

Satisfaction with publicity 
materials and directions 

Satisfaction with convenience 
and helpfijlness 

Satisfaction with timeliness 
of service 

Satisfaction with quality of 
technology ' - 

Satisfaction with value of 
bibliography 



1 
0 

1 
0 

1 

0 

1 

1 

"0 , 

1 



"H/'ariables 1 through 4 and 
sheets,. Variables 5 and- 7 
tion Questionnaire. 

^See Table 27 for conplete 
of siibscales. 



6 are recorded by search analyst on data 
through 12 indicated by client" on User Evalua- 

list of satisfaction items .and construction 
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^ - - , TABLE .26 - • ' 

PROCESS AND OLriHJT VARMLES USED IN SEARCH ^SERVICE. EVALUATION 



Variable"^ 



Item Description and Coding 



Number 



Name 



Description 



Coding 



4 
5 

6 
7 
8 



9 
10 

11 

12 
13. 

14 



CONTACT 

SYSTEM 

TURNARND 

V 

CNCriME 
STRATME 

CitPRINT 
PRICE ■ 
AMTLRDTC 



SATIS6^ 
SATISl , 

SATIS2 

SATIS3 
SATIS4 

SATIS5 • 



In person v 
Other \ % 

SDC* ORBIX:_^ 

LIS SlaLOG. ' ^ ' r 

^^Turnaround tinie in days ^between 
receipt o£ request and online 
searching 

Connect- time^ in minutes 

Interview and strategy time in 
minutes 

Number o£ citations printed 

Price charged in dollars 

How much did you learnr about 
your topic as a result q£ 
the seexchl 

CI) ..^ Nothing or veryllittle 

C2-D • '"Some 

(3) * A great deal , 

. CX^eral^ satisfaction 

Satisfaction with publicity 
materials and directions 

at 

Satisfaction with convenience - 
and helpfulness 

Satisfaction with timeliness of 
service 

Satisfaction with quality of 
technology 

Satisfaction with value of 
bibliography 



1 
0 

1 
G 



1 

ERIC 



■^Variables 1 through 7 recorded by search analyst, on data sheets. Vari- 
ables 8 through 14 indicated by client on "User Evalulation Questionnaire. 

^See Table 2 7 for conplete list of satisfaction items and construction 
nf subscales. 
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■ ■ , TABLE 27 
ITEMS USH3 IN' SIX SATISFACTION SUBSCALES 
■ ■- / ^ . 



Question: M^i|^tJ^ satisfaction 4th the following ele»^ts 



SATISl - Publicity Material and^ Directions - 

a. Convenience of arrangements and adequacy" of directions 

Accuracy^ and comprehensiveness of publicity materials 
c. Adequacy of directions for ordering "materials 

SATIS2 - Convenience and Helpfulness " . . 

, a. Convenience of arrangements and adequacy of direction 
b. Helpfulness of search analyst 

SATIS3 - Timeliness of Service 

a- Time taken to ..deliver bibliography - 

b. Time taken to deliver materials from EDRS 

c. Time taJcen to deliver materials from EISO 

SATIS4 - Quality of Technology 

a. Length of bibliography 

b. Readability of microfiche copies 

c. Availability of microfiche .readers 

SATIS5 - Value of Bibliography and Materials ' . 

a. Bibliography itself . ^' 

b. Materials located via bibliography 

... . ^ 

SATIS6 - Overall Satisfaction ' 

la, 2b, 3a, 4a, 5a " / . 



Note: Scale scores were adjusted for the number of items by dividing 
- total scale scores by the number of items. Scale: Low = 1- 
■medium = 2; high = 3; N.-\ = 4. ^ 



The final input .y^ri^le , SYSTEM, refers to the ^omputer 

3earch service * that^was used in executing- th6 search: SDC ORBIT 

or LIS DIALOG. Although this variarble is -a system rather'than 

' user input, it Was'- included' in the analysis because dvifferen- 

ces between the two influenced the selection of the system 

used in conducting .searches . In particular, the direct mail 

Service provided by SDQ, which allows the ^computer-produced 

bibliography to be sent directly to the' client from SDC, meant 

that this service was often preferred for users located some 

v>distance .from Toronto. ' ' " - 

User satisfaction with various aspects 'of the search . 

service weir^e mearsured by five subscales (SATISl to SATISS) 

-consisting of scales t?hat included saTtisf action^with publi- 

city materials and direct ions tSATISl) , with convenience and 

he-lpfulness (SATIS23 , with timeliness of Service CSATIS3) , 

with the quality . of technology CSATIS4) , and with the vali:^e 

of the bibliography> CSATIS5) . ^""^ 

Individual items used in constructing "subscales are 

reported in Table 2 7 and- include those used in the scale used 

to measure overall satisfaction CSATIS6) , The lattjer scale 

used five items selected from the subscales ; not all items 

were included becaxise many clients had not been exposed to ■ 

all aspects , of ^the service'and therefore hjad failed to res-. 

pond to some items. Instead," only those items referring to 

matters to which all clients had been exposed were included 

in the scale. ' * ^ " 

f Reliability coefficients for^ the six scales were .80, 

.62,;. 78, .27, . 79 , and ..80 respectively. Overall, these 
■. * 

coefficients _aiLe_suf f icient ly high for the scales to be used v 
in -assessing the satisfaction of EISO's users with the service. 
The fact that all input variables ;are dichotomies calls ' 
^or special care' in interpreting correlation' or path coeffi- 
gcients between these and other variables. For example, a 
positive correlation between location and organization would 
imply that users from Metro Toronto (coded 1) tend to be 
associated with OISE C^lso^ coded 1}, whereas those from out- 
side Toronto tcoded 0) tend to be associated with other insti- 
tutions (also coded 0).^ Where'^the second variable Is * continuous 
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as m the case of sa-tisfaction, a positive correlation implies 
high ■ satis fact: ion is associated with- the' trait coded l^'and low 
satisfaction with the trait coded 0. In the case of the re- 
lationship between, say, location and satisfaction , va positive 
correlation would imply that users from Metro- Toronto (coded 1) 
were more satisfied than those from outside Toronto Ccoded-O 
A total of 14 variables are used in the second , analysis 
which is'devoted to the study of the effect of the search pro- 
cess on user satisfaction -(Table '^). Seven, are process 
variables, one is an intervening variable, and the remainder 
axe the satisfaction variables that have been previouslyr 
described. " ' . 

■ The -first * two variables in the second analysis, method 
of contact (CONTACT) and system used ^SYSTEM) , are dichotomies 
and coded as 1. ajid 0. 'As noted earlirer, care must .be taken in 
interpreting correlation and path coef f icieijts computed between 
these and other variables. " . ; 

Turnarround time CTURNARND) is simply the number of days ' 
that elapse between- receipt of a -search request and execution 
of the searxh. ' In practice, thJe date of each transaction is 
recorded, and the difference between the two is calculated at ' 
the time data are analyzed. ^ - ^ 

Connect-time in minutes CCNCTIMEO and the number of"^ 
citations printed (CITPRINT) , data listed automatically at 
the end^of each search',* are transcribed to the data-sheet 
used for EISO\s records by the search analyst. The analyst. 
i'S^also responsible for recording" the -estimated amount of ^ \ 
time^ spent in question negotiation and -planning the search 
strategy CSTRATIME) . Because of the complex and-^ iterative 
nature of these tasks, the figures recorded^ for STRATIME are 
prob-ably accurate only within ±15 minutes. 

a \ The^ price (PRICE) charged for each search^-is recorded' 
in Ca^a^dian dollars. While this is striaightf orward, the 
pricing policies in. effect during the period data were co*l- 
lected were- not. Four different modes of charging were 
used: 

!• .free searches allocated to target groups to 'encourage ' 
.'trial use of the service. 
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2. $20 flat rate for clients from a target area located in 

Northern. Ontario,' ■ * ' " 

3.. $3"0 flat rate for searches without an* excessive number 

of citatidns, . • " ^ ' - * 

$,30vplus a surcharge of- $.10 per citation fbr searches 
. ^ with ,an' excessive number of icitations. ' " 
The. policy cni charging for an' excessive .numtrer of citations 
was applied in a' flexible manner by the search analyst. Typi- 
cally, it was app4j.ed whenever the combination of connect- 
^ time, communication, "and print costs exceeded $45.00. It 

must be emphasized that pricing policy has a direct effect. - 
■ On 'the 'types of relationships one can expect to find. Because 
most EISO clients received searches for either $30.00 or 
nothing during the period under consideration, any effect of' 
PRICE on other variables will be interpretable as -the effect 
of charging $30.00 as opposed to providing free service. • * 

The amount a client learned about his* 'or her topic 
CAMLTRDTC) as a result of undertaking a search was tapped by 
. the s^aightforward question listed a^ variable "8 in Table 26. 
"^s. with other one- item variables, it is not possible to com- 
J>ute a rel-iability coefficient for, this measure. .However, 
its validity i^as confirmed by. correlating responses on it with_ 
. questions a^out the number of original documents read, and the 
' percentage of information that was new. -to the client. Correla- 
tions coefficients were .36 and . 40 respectively , both signifi- 
cant 'at the. ^05 level-, 

• Sociological Input Model. 

-:.Who are EISO' s users and how do*es ; their identity- affect 
their levQ^ls of satisfaction, with 'the service? As suggested 
^above, these are important questions which reflect . the ' success 
-..of the service, in achieving its goals. " By implication, if one 
kind of client is less satisfied than another, changes in the 
service might be necessary in order to raise their level of ' 
satisfaction. ^ . 

^'^The question^as to the users' identities is answered 
here in terms. of the two sociological concepts noted earlier, 
role and status. We view these as' characteristics that clients 
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bring wi^h them to their encounter with the search service, 
and. as such, represent inputs to the system. Since status is 
Itself in large part' an attribute associated with a given 
1 rjzoley it_-X5- di££icul-t- to- measure- these - two- -aspects- sep^ratel-yv- 
\^ost of the variables we have selected reflect, 'to ^ne degree "r 
CMT another both the role and status of the individual client.- 

Description of Model. 




educator working or studying, at some location in Ontario, The 
user^s geographic location (LOCATION) and sex (SEX) . were viewed 
as two background variables ^of^possible importance. Location 
is a. f'actor ^f considerable policy importance since onfe o£ 
EISO's major purposes is to make the resources o£ a major 
research library available to educators throughout the pro- 
vince. To be sure current limitations of automated retrieval- 
systems make this a distant goal ^ yet the quick. retrieval of 
both references and original documents in RIE, CUE, "and 
ONTERIS does provide access to a large and' important body of - 
knowledge. While/the sex of a' client is not, in and o£ itself, 
of primary • importance , it is a factor known to be related to^ 
, the., role of /prof essional educators; in particular , '"i^elatively 
few women currently hold adminirs-trati ve positions 'within 
Ontario's educational system. Thus, sex' was viewed as a 
background variable that should be controlled. 

< A client's organization CORGANIZN), professional role 
■ (PROFROLE;) and purpose in conducting H:he " seaTch [PURPOSE) 
were viewed as the major characteristics defining his or her-' 
role. Though the coding scheme* used in this analysis greatly 
simplifies the original oategorization scheme , we^ originally 
determined whether or not a user was in any one of 29 dif- 
-ferent types of organizations or units; e.g., preshcool ,\ . 
public schGfel board, separate school board, private school. 
College -of Applied Arts and Technology,, faculty, of education', 
etc. Professional roles were classified into 14 categories; 
e.g., administration or supervision, teaching, pupil person- 
nel services, research, etc. Finally, the , purpose of the . 
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search, which we saw as an objective in large part determined 
by the individi^al '-s organization^ and ro^le , was initially place< 
in. one of eleven categories; e ..g keeping 'abreast of the fiel< 

—<.^3.ss assignment-,- prepaTration- of -a •-bibli-ogi;aphy , "curricuTum 

development, etc* 

• To illustrate .the connection among PROFROLE , QRGANIZT^I, ^ 

and PURPOSE, consider two typical users. One might be a fjurll- 
time graduate"^ student at a faculty of education conducting a 
lite.raturea search for a class 'assignment; the other might be 
a senior administrator- in a pub Tic school board seeking in-: 
formation about curriculum guidelines. ' . 

Another inpi^^ var^&ble related to the servicers acti 
ties is the partivCular search system used (SDC^s or LIS^s), 
While not a sociological va-riable, this characteristic (SYSTEM) 
is included as a factor of possible importance because one of 
the-systems SDC, offers^ the capability of .mg.iling bibliogra- ' 
phic output directly to the client. For those clients in 
remote locations, therefore, SDC may^be a preferred service 

in terms of their satisfaction with the servicers turnaround 

- . - • ■ - 

time . 

' Finally, the. output variable of interest is satisfaction, 
which was measured -by 12 */i tem'^. rela.t ive to 5 distinct aspects.^ 
of the service": .publicity mat^^als and directions (SATISl) , 
convenience aiiiH. helpfulness CSATIS2>, timeliness of service 
CSATIS3) , g.ukiity of .technolo.gy CSATIS4) , and value of the"^ ^ . 
biblio'gr'aphy (SATI.S5) .... A subscale of 6 items was also derived 
to measure overall satisfaction CSATIS6). ^ 

Organizing the variab ' described, above into a formal 
path, model * results in the diagram pictured in Figure 4. 
LOCATION and SEX are background variables whose relationship 
.to one. another remains unanalyzed, but whose effects on other 
variables are considered* Location may affect satisfaction 
both directly and indirectly. One indirect path is through 
its influence on the system-^used. The second effect is via 
its influence - on the rtype of organization from which the 
client come-s. Sex, too, may" affect satisf acti&n both directly 
and indirectly. ' The indirect effects may be through the type - 
of .organization or the individual * s prof ess ional role^: 



- .Organization stands as a key intervening variable between 
the: background variables and satisfaction, on which it may have 
. a directs effeclr.^. However /it may also, have ^i effects 
_ ■ -}h^ .. P ^° ss i onal r ol e _ a^^^ ,o f. .:the- 

s-earch. Similarly, professional role may affect satisfaction 
: directly or through the purpose of .the seaVch-, while the latter 
variable .is postulated to have'.only^ di-rect. effects. 

It may be noted that other paths than those noted are 
possible; howeveV, we are including only those which seem 
logical given known relationships. and "the meaning of the 
variables-. Thus 1 for example , Ve have omitted paths - connecting , 
- say, SEX and SYSTEM.' It should also be noted - that, where direct ' 
...effects are stiggested, these effects are residual to the- 
• effects through other variables. .Thus, the*'path connecting 
organization and satisfaction represents the effect on satis- 
faction of the individual's organizational status after the : . 
effects of the person ' s prof es sianal. role, and' j^urposfe ,in con- 
. ducting the. search have been removed. . All residual ydriation 
which^is not accounted for by any of the variables in the model 
is. as^ed^ to -be caused" by external variables, represented by ' ' 
the ^i's, ^ , ' . . 

The problem for this particular analysis, then,, is. to 
. test the validity of the 'inferences about cause and effect 
relationships suggested by - the path analysis model .depicted 
in Figure 4. While confirmation of the -relationships des- 
cribed by the model- does not prove t>he existence 'of the implied 
causal relationships, it would p.roVide evidence supporting 
their plausibility. 

■* . . . o 

■ ■ . ■ . . . . - . 

Findings . « . . 

In this section, 'we shall proceed from the general to 
the specific, presenting first the findings for the path 
analysis in which overall satisfaction (SATIS6) is the- d^Ren- 
dent or. output variable, and then the; results for the _ five!., 
satisfaction subscalres. ^ Table 28. lists the cbf relation coef- ;'• , - 
. ficients,':raeans,- and standar.d '-deyiations for the seven' v^ri- V 
ables that appear in the path diagram in Figure- 4.' The means 
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. TABLE 28 ' ' ' 

CORRELATION MATRIX, MEANS, AND STANDARD DEVIATIONS 
FOR VARIABLES IN SOCIOLOGICAL INPUT MODEL EXPLAINING 
OVER^L SATISFACTION WITH SERVICE 



Xi LOCATION 



X. 



X2 SEX 



,095 



X3 ORGANIZE . .511 



t .. . 



,087 



:X4 PURPOSE 



.430 



197'*' -.434'^ 



X5 PRDFROLE; -...i35§ . -.309,"^- -r348"^ 



294 



T 



Xg SYSTEM 



X7' SATIS6 



-.093 



-.050 



,00-4 



,024 



.007 



,053 



-.059 
.035 



.030 
.081^ 



,094 



.Mean 

Standard 
Deviation 

N = 162 r 



.531 .364 .352 .469^- .327 .154 ' 2.59 

'^^^ -485 .479 .501 .471 ;362 . 0.37 



' +p < '.01 

■■§p < .05 
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for the sociological background variables need some interpre- 
tation since the variables; themselves are dicTiotomies 

The mean of .53 for .LOCATION indicates that 5 3% of the 
users in the sample were from Metro Toronto and, conversely, 

.....that .47%_..wexe_ ...from_. outride... Tor on to-, Similarly-,- the^^ means- foT^ — 

SEX, ORGANIZN, PURPOSE, PROFROLE, and SYSTEM can be . interpreted 
in terms of percentages: 36% were students or staff at OISE; 
47% of^all se,arches were for professional as opposed to aca- 
demic purposes; 33% of the clients held administrative posi- .. 
■ tions; and 15% of all searches were conducted using SDC. The. 
^ mean of 2. 59 (on a scale from 1 to 3). for oVerall satisfaction 
indicates the average user was highly satisfied. 

The zero-order correlations j among the five sociological " 
background variables are see'n, for'^the most part, to be statis- 
tically significant.^ In contrast, neither SYSTEM nor SATrS6, 
overall satisfaction., have and statistically significant . 
relationships with the other variables included in the analysis. 

The path analysis in Figure 4 shows that the pattern' of 
correlations- exhibited in the correlation matrix are present 
even when all of the background variables are entered. 
^Tracing the various paths, we see the following telat-ionship 

LOCATION has a very weak, non- significant positive cor- 
relation with SYSTEM, indicating' that SDC was not used more' 
often; for clients distant from. Toronto Given the coding 
system, a negative relationship would have_ been, indicated. 
Location has a non-significant negative correlation with satis- 
faction. What - slight, relationship there is would- suggest .- ■ 
greater satisfaction among t^ose users outside the immediate-- 
■.metropplitan area, where .E ISO. is: located. FinalTy, location 
■does •have a strong pds itive- .coVrelat loh with^ b.rgani zation," • ". 
confirming that most of OISE's 'staff and students .who use." ' 
the system are associated with the main campus in Toronto 
rather than one of .its nine fiel^d ■centres- 

other prior variable contained in the ^path 
TOodel^in Figure- 4 . exhibits only one significant ."relationship , ■ 
that- with PROFROLE. 'This trelationship imp^lies^that EISO's " 
womeri clients tend to hold non-administrative-^sitions, and 
that^ its male clie^i-ts - tend to hold admijiistrative positions-.- . - 



At the-nexf stage of analysis, ORGANIZN i^, seen to have 
significant relationships w.ith both the purpo-se 'of the search 
(the negative path coefficient implies, those outside OISE- are 
; more likely .to . request searches for professional rather than 

_ ; academic purpos e.s indrv-idual-- - 

; (the nega.tive relationship -im-pl^^haf OISE users 'tend not 
to be administrators, while. pro^^^^,lS,client^ hold 
administrative positions);.. ^■^^.^^^fe^NI2N -li-^s a residual 

^ positive ^ffedt on satisfact^pn^te^, tft"^^ 

due to SEX, LQCATIpN;^'^njRPOSE-/ and-PR.0i^;ROLE. . '••^ 

The last stage of the path' diagram -iuvo'lves SYSTEM ^ 
PURPOSE, ^nd PROFROLE. - . sYSTEM is s^een to have.no statisti- ' " 
cally significant, effect on 'satis faction.- ' Similar .conclusioris ' - 
.hold for^.PURPO.SE -- academic. -and .non- academic users are equally 
satisfied -- and PROFROLE - -adminis tratots ' and non- admini s trators 
are equally satisfied. ' , ' ■ 

Turning now to the path . analyses. "for ^he satis faction sub-, 
scales reported -in Table -29.-; we note that thai; patt^ern of path - 
coefficients is very similar to that observed foi^ ''"S verail satis- 
faction. In fact, all coefficients not involving a satisfac- . 
tion subscale are in f.act estimates ot .the. same coefficients 
■reported under the preceding analysis. They differ only 
.-because, of the different sizes of the sabsamples for which ' - 
■,C9mplete data were available. . 

^ ; Cohcentratdng^on the- satisfaction subscales we note two 
significant corre tat ions . Since, among the 36 path coef f ici^en ts% 

would expect about ;this number -to statistically significant 
at the .05 level by chance alone, even .when there were no real 
relationships, we, should;, not place, much tveight -upon 'them. - The 
two-'p.a>hs w^ . refer to are be tween . PURPOSE ■ and- SATISl , satisfac- 
tion; wi-th..3^15licity mater ials: ^ildV di^ectiK^hs . This^ ^ Elation ' 
ship would_vLmply slightly greater satisfaction with"' these 
among users requesting searches for non-academic p^^rposes. 
The second significant re^lat ionship between LOCATION and ' ■ 
SATIS5, satisfaction with the;value of the bibliography-, and , 
materials, is negative. This would imply that users from 
outside are more satisfied with the end result of sear^ches . ' 
than. ;are those , fro.iii- in;side:'Tor"onto . . * ' ■ 
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•■ ' . • TABLE 29 

PAHj COEFFICIENTS F(jR' SOCIOLOGICAL INPUT MODEL E](PIAINING SUBSCALE SCORES 

FOR SATISFACTION Wim' SERVICE 

""^^ ' ~ ^ ^ Variable 

Path Statistic — '■ 

i\m Skmi SATIS3 SATIS4 



I. 


LOCATION-SK 


^12. 


• .11 


.' .16^ 


-.06 


.11 


■2. 


ORGANIZN-LOCATION ' 


> 


, .52'^ 


.55^ 


.25" 


.70^' 




ORGANIZN-SEX ' . 


. P3'2 


.05 


.06 


-.01. 


-.04 ■ 




PURPOSE-ORGANIZN , , 


hi 


-.36^ 


-.39^ 


-.36 ■ 


-.33+ 


5.. 


PURPOSE-PROFROLE ■ 




.11' 


■ .15^ 


-.07 


.27^ 


6. 


PROFROLE-SEX 


'P52 . , 


-.32^ 


-.28+ 


-.42^ 


-.35+ 


i: 


PRDFROLE-ORGAN'IZN 


,P53 


■ -.31^ 


-.34+ 


■:-.,39+ ■ 


-.33+ 


I 


SYSIEM-LOCATION 


P51 ' 


.06 


M 


.18 


.02 


.9. 


SATIS-LOCATION 


: h ■ 


.11 


'.11 


.37 ^ 


■'-.11 


10. 


SATIS-SEX 


P72, 


.08. 


-.05 


-.21 


.11 


11. 


SATIS-ORd'IZN 


, ■ ■ P73 


.03 


, :ii 


..06 


.07 


12. 


SATIS-PURK)SE . ' ■ ■ 

0 


. 'P7^ 


:/ .18^ ' 


■ ■ .13 


.19 


.02 


13. 


SATIS-PROFROLE 


' , P75 


-.08 


> -.06 ' 


■-.31 


^-.07, 


14. 


SATIS-SYSTEM' ' ■ 


P75- _ ■ ■ 


. ' '.01, 


■,, V.ll , 


,.07 


".12 




■' ■ '. ■■■■ ' ■ 
n • ■ ■ , ■ 




■ , 171" 


■: 182 ' ' 




. 75 



I ■ . ''. , , ' . . 2.57' '■2.7;7, ^2.4^ ' 2.51 



Discussion- \ 

' ~ , \ 

This path analysis shows that EISO has satisfied both 
of its major markets with equal effectiveness. One of tl>fese 
markets, the professional market, is-, composed especially of 

- - ma-le- admin is t r a to r s " "To c a t"e d^'th rough out"" 0 o V ~ t li e '6 1 her 

academic market, is composed of students and academicians. 
This group has a higher percentage of~ women than the other, 
and is concentrated in Toronto, especially at OISE. 

The fact that the sociological ^dnput model fails to 
explain different levels of client s a-lti s fact ion can be taken 
as a mark of EISO's success. Housed as it is in an academic 
library at OISE in Toronto,, one would expect a tendency for 
it to serve its immediate constituency best. The very lack 
o£ such a relationship indicates it has been successful in 

- satisfying the needs- of professional educators with equal 
effectiveness. ' ' . 

It should be emphasized that EISO is effective. The 
mean score for the clients overall s a1»is f ac ti on was 2 . 6„ oh a 
three-poirit scale. The subscale means, in Table 29 show that 
users are also highly satisfied with EISO's- publrcxty 
materials and directions (SATISl) its conve.nience and help- 
fulness (SATIS2J , and the quality of its' technology CSATIS4) . 
The users reported a moderately high level of satisfaction - 
on the remaining two subscales , timeliness _of service CSATIS3) 
and the value of the bibl iograpliy and materials CSATIS5>. 
Given that the- time liness of service was in part dependent 
on delivery of materials from EDRS in the U .S. , we can even' 
discount, to some extent, one of these two.. 

; ' The • fact that the sociological input model has failed 
to explain different levels of satisfaction does not mean the,- 
small variation observed in these variables cannoi: 'be explained; 
there may be other models that are more effective at thi-s task. 
With this in. mind, we now describe the- second model , which con- 
side'rs process variables which are more directly under the 
control of the EISO search service. 



Process Model - . 

As with. the sociological input model, the process model 
for explaining levels of -user -satisfaction represents a sub- 
system within" the larger evaluation framework, and involves 

or^ly. .a_f ract ion_ of the _data._colJLected_ ._lCey process -variables — 

are '.the method of contact (CONTACT), the system used to ' . 
search the data base (SYSTEM), the length of time required for 
t}t4 search interview and to plan the search strategy (STRATIME) 
the connect- time (CNCTIME) , the number of citations printed 
(CITPRINT) ,- and price (PRICE). .All of the.se variables except 
the method of contact relate directly to the cos^. of . operating 
: the service or the price chal-ged., The method of initial 
contact describes the actions of the" client in playing the 
role of request9r. Two important variab les , the data base • 
being searched and the ia^ntity of the search analyst," have' 
been eliminated in this list of process variables since they 
are being .held constant by ^limiting the,,.analysis to searches 
of the ERIC data-base conducted by one search analyst, 

*The output or- dependent -variables at issue are the 
.. amount learned by • the Client about the topic searched (AMTLRDTC) 
and,, his. or her satisfaction (SATIS). In fact, satisfac.tion was 
measured with 12 items which were grouped to form, five isubscales 
CSATISl. to SATIS5). measuring client satisfaction with different 
aspects of the- service; , a subset of five of these items were 
added to ^^;give an indication of overall sat isstaction 'rSATTS6) 

Description of Model ^ / ' 

Using the temporal sequence of events provided by' the 
•systems- framework, we can order the process' and output vari- * 
-ab-les and thereby refine the statement of ,/the problem -into a " 
series of que'stions about pair-wise .re^latidnshipsT^ Method of 
contact should clearly be the first variable, followed sequen- 
tially by strategy time; turnaround time, selection of. a sys- 
tem, -connect- time , citations printed, price, amount learned, - 
and finally satisfaction; This sequenc^ does not imply' that 
each variable causes the one that follows; however, within ~ " 
the sequence, there are a number of plausible cause -effect 
pairings. /" '\ . 
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Method of contact may have a direct effect on both 
connect- time and strategy-time since if a person is present 
in person far more discussion and searching is likely to occjjr 
than if contact is by telephone or letter. Connect-time and 
strategy- time, in turn, could effect ^the number of citations 
printed since' more time ' spent on these probably results"' in a 
longer bibliog-raphy . The number of citations printed might 
affect both price, if a' variable pricing policy is used, and 
the amount learned by the client, since a longer bibliography 
may contain more information. And, bo/th price an'd amount 
learned ■ might affect satisfaction. ■ 

It is unlikely, in contrast, that the system used or 
turnaround time will affect any of the process variables , 
thc^ugh both might well affect, satisfaction : turnaround time 
bex:ause long delays might reduce sat isi^alt ion , and system 
used because one might be more effect ive%han the other at 
retrieving relevant references^' If the latter is the. case, 
then the system used may also affect the'^amount learned by "^^ 
the client. Finally, the amount a client learned might also 
be "positively influenced by the time s.pent in negotiating and 
planning the- search strategy, independent of the" bibliography .- 

The problem for investigation thus becomes testing the 
validity of the preceding inferences about- cause and effect 
relationships between different, pairs of variables . The par- ^ 
ticular method selected for this purpose is path analysis. 

the path -diagram" for the relationships' described above 
is disp^J-ayed in Figure 5. Both temporal sequence and cause - 
effect relationships are indicated • by the flow o"f, the diagram , 
from' left to right. Following the . convbntions- of path^-dia- - 
grams ,- those va^r^bles with no predecessors'-^ are aligned to 
the left. The -diagT^am^is helpful in-showing the flow of . . 
action 3.s a search is executed, and in identifying the rela- : 
taoriship's being- investigated. >he path .coefficients noted on 
th~ecdia:gram are discussed later, but it is worthwhile noting 
the introduction o^f hypo tTiet ical variables (the e'sj which 
represent external variables whose e^fecfts are not explained 
by variables in the model. ■ ■ . ' ' ' \ V . 
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•Findings - 

In this section we shall proceed from the general to 
the specific, presenting "firs t the findings for the path 
analysis in which overall satisfaction CSATIS63 is the depen- 
dent or output variable;, and then the results for the five-" 
satisfaction* sub-scales. 

.^\Table 30 .lis ts ' the,. correlation coef f icient^ , ::mearis' , and 
stanaafd deviations 'for the nine variables that appear in the 
path diagram in Figure 5 * ' Sever al of the means need interpre- 
tation to be understood,. For CONTACT, X = .53 implies that 
53% of all clients contacted the 3ea:r'ch analyst in^.'person; 
similarly,, the mean :6f .19 for SYSTEM implies . that only 19 % ^ 
.of all searches were done using SDC 's ORBIT while the remainin 
■81% were carried out using LIS's DIALOG.' ' ^ 

The average delay in running a search was 4 . 3 days , but . 
the actual search process took about 70 minutes, including 46 
■miriates - for interviewing and ;planning^ the strategy, and 24 
minutes for. running the search. The average bib 1 iography 
included- 104 items , and cost the ent $28., though this : 

average is of questionable Use, since most clients either re- 
ceived the search free 69%3 or'paid.$50 C82%).: The average - 
client learned /'something'* about the topic searched Csirice ■ 

at *2--2 the mean for AMTLRDTC i*s near the .middle of 'the three- 

■ " - ' ' • ■ ' ' . ' ■ '■'.■»'- ^ " - . ■ ■ ■ - ■ . ■ . ■ . • 

pjcrint' s^^ale) and expressed high overall satisfaction. . (2 .^5 on 

a scale, .of pj* .. ^ ■ . ' ■ . ■ ' ■ . " ' • " ■ 

. We will not anaUyz^^e. -the -cbrrela^ in detail since 

that is the purpose of the path ^analysds . However-,- we would 

i» ■ . . ■ ■ . , '■ . , ■ 

note that initially, only the amount learned about the topic. 

has: a sigf^^f icant zero-orde?- correlation with overall, satis- 
faction . / 

■ Path coefficients for analyzing the causal e^ffect of the 
sreVeTi proo^ss^ variables on the two output variabl'es, AMI^LRDTC 
and SATISd ,^ are included in Figure 5, where each causal path 
is indicated by an arrow. The most informative paths are 
those that can be traced from CONTACT through CNCTIME and 
STRATIME . to GITPRINT, .AMTLRDTC; -and PRICE, .krid finally to 
SATISd. - • . 
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Both ;CNCTIME and STRATIME are 'positively influ^hced by 



the personal contact -with 
on the niimber of- citations 
through connect-time . The 



the client, hut the effect, of this 
printed, -is^ relayed almost solely 
amount learned about the topic 
searched CAMTLRDTC) does not appear .to be affected by any of. 
the process .variables . The small negative coefficient -between 
it and SYSTEM^ which might imply that SDC was/ .less effective 
.than LIS (as our search analyst fervently believes) , is not 
statistically s(^n±ficant. Similarly, the positive coeffi- 
cient f rom CITPRINT , which would imply-more is learned from 
long bibliographies, is not statistically significant . 

CITPRINT is a statis^ticaliy' significant determinant of/ 
PRICE as would be expected given the" pricing poTicy in which 
charges for an '^excessive*' number of citations are made. 
Other variables which might|?^f f ect price, such as being in 
one of the target^ groups receiving ^ free searches , ^are- excluded 
from the model but are« represented by e^. I 

Finally, five causal variables impact upoi=i bver'^11 ^ 
satisfaction (SATIS60 but only two of these relationships are^- 
statistically signi'f icant : those "for AMTLRDTC and PRICE/ The^ 
.'first haus a. positive effect, with those :clients who learned 
more about their topic having a more positive viewpoint. But, 
controlling for CITPRINT, AMTLRDTC^ SYST-EM, and TURNARND, ,- 
^t:hose who paid a highe less satisfied overall . 
That is, all other things being equal , the clie a 
higher-^ rate would report lower overall satisfaction than one 
who paid a lower rate. Thus, liffiough- price did not; appear as : 
an important variable in -the zeroVorder correlations , it does 
appear as. a ^determinant in- the final analysis". Of the ^^emain- 
ing variables which we: l^ought might affect overall satisfac- 
tion, namely' TURNARND / SYSTEM and CITPRINT , a po^ss ible ' re Ja- 
tionship is indicated only for the last. Taking " the three 
branches from CITPRINT together, it might be suggested that 
it^does affect satisfaction^ with its positive effects being 
felt both; directly ^and indirectly via the path through' amount 
learned,- and its: negative effects felt via the path through 



. -Table 31 lists^ the path ,,Gpe££icients for the pr^qcess 
models in vhicli hot onl^r o'ver^;!! satisfaction is the dependent 
variable -of ^nterest,. but aJLso the subscales measure specific 
Characteristics of the-.search service, SATISl, through SATIS5. 
In the table, paths are named with the effect variable, first 
arid the causal variable second • ^ The subscripts -also reflect: . 
this ordj&4ring, e.g.; p^ ^ is the .coefficient measuring the 
effect of Xi .CCONTACT) on Xi, (CNCTIME) . An exception to .this 
rule occurs when the correlatioris remain' im as between 

the pairs of exogenous variables Xi to X3. For these variables 
tTie zero-order correlation coefficients are presented. 

Mi^ans , standarxi' deviations , and sample sizes are also 
given in Table 31.; Sample s izes vary according to" the satis - 
faction subscale being, considered since not all clients were 
exposed to all facets, of the service, and hence did not respond 
to some evaluation questions^ ■ • * 

Path coefficients for^ the first 11 paths ^or the five 
satisfaction subscales are-, in fact , estimates of the same 
path coefficients . already considered in the diagram expiaining 
overall ^satis'f'action. Any vacation from, earlier est^-mates is 
due to variat^ion in the samples on which the estimate's ai:.e 
based. In fact, the new estimates are quite similar to the 
earlier /ones, except for -the sample size of 31 on which esti- 
mates for path coefficients leading to SATIS 3. are .based/ This 
is to be expected given the smail sample size, and serv 
a' warning against placing too "much conf idence "in ^^e^tiniates 
drawn" from this subsample , <. ^ . / 

The*^ lasy^t five coefficient.s in each column of Table 31 
do present. new p^-th coefficients which measure the effects of 
SYSTEM, TURNARND,. CITPRI.NT, PRICE , and AMTLRDTC on the . satis- 
faction subscales. ; / - 

Starting with SATISl, the degree of sat isf action with 
publicity materials and direct ions , we note ^than only PRICE 
seems to have an effect, and , as before,^}ie effect is nega- . 
tive . > A similar conclusion holds for SATIS2 , ! the convenieii^ce . 
of the service and helpfulness of its staffs 

CITPHiNT, PRICE , and AMTLRDTC ■ all have signifxcant . 
effects on the satisfaetion of the 31 clients >rho- respoi^ed . • . 
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to SATIS3 which measured the timeliness of the service , includ 
ing document delivery from EDRS and <E ISO. ..These coefficients 

•are similar to those noted for overall satisfaction, but are 

■ ■ ■ '* ■ ^ 

somewhat longer. . . / . . . 

In contrast to the preceding case, only TURNARND , and 
SYSTEM 'apparently affect SAT I S4, which, tapped client satisfac- 
tion with the quality of the technology ^ as measured by 'the ^ 
length of the bibliography , readability of microfiche;, and- .v. '^ 
availability of microfiGhe readers-. 

Satisfaction witW the "value of the bibliography and, 
materials located ' (SATIS5) was apparently affected by only 
one variable, the amount learned about the. topic. It is 
notabie that, fo;r" this specific attitude object the value 
rof th^ bibliography and inaterials .located through it - - - t'he ; 
negative effect that price had on other types of satisfaction 
disappears. . / • ; • 

Discussion^ 

How: successful is the process model at explaining user 
satisfaction? We are embarrassed to- admit, having put con- 
siderable effort into its development and testing, that it is 
only slightly more; effective than the input model at doing the 
job for which it was intended. FortunatJe^, it does provide 
.additi'onal information tFiat increases i^s overa^-l usefulness . 

As far as satisfaction ^-s concerned only one variable 
seems to have a consistent effect, namely price . The negative 
ef:^eet of P^ic^ satisfaction was apparent with overall ' ' 
satisfaction, and satisfacti'On^wd'th particular comp.onents of 
the service:, publicity - materials and directions , convenience 
and helpfulness,: and timeliness .:\ It>did liot affect satisfac- \ 
tion with the, te^chnology or materials produced. In interpret- 
ing -this differential effect, of price ,/ it must be recalle'd 
that 9%. of all* users received free searches, and .most others 
paid $ 30 OT^: more • It would appear , then, that when indi-yi-. 
duals are' paying for " a bibTiographic search , they expect- more 
in the way of ^service . thalri - they would if they were receiving-^ 
it" for free.- • Yet, it i-s ^important to^note that their-, expec- 
tations with regard to service do not carry^'.over ;to. their 



assessment o£ the value o£ the goods, which they apparently ^ 
assess independently "of amount- paid; - . - ... . ^ 

We are ..at a los^s. to . account fox the three /other signifi- 
cant •relationships between process variables and satisfaction 
scores (SYSTEM and TURNARND with SATIS4, quality of te.chriology; 
"and CITPRINT with SATIS3, timeliness of service) . We are. * . 
■si^clihe'd:- to ascribe them to anomalies, in the" "smallex sainples 
involved, or to. extraneous variables not in the arialysls. 

- .There is ji; of course , another Variable, which is related , . 
to various' satisfaction scores with some consistency, th^ 
amount' learned ablbut the topic searched CAMTLRDTCO • This 
.variable is an output variable in its own right, but inter- 
venes between :satisf action and all process variable's excepts ■\ 
price, - It is positively relat.ed/to' overall satisfaction, < 
satisfaction- with, timeliness of .service , and most important,' 
satisfaction with the value of the bibliography and" materials • 
This last relationship, heflps to ^ validate the satisfaction items, 
since one would expect a person who learns^ more than another 
from a bibliography to. be ^niore satisfied with it. But more 
important, it highlights the idea that the client is motivated . 
to learn,' and that if this learning does nat occur, dissatis- 
faction will result. 

Though the amount learned has .proven to ^ be an imlportant: 
intervening variable explaining client satisfaction, none of 
the process variables helps to account for the extent of: 
learning that takes -place. This failure to account^- for^jvaraar 
tipn in learning is an important finding in^ and of - itself ,- 
since the* following possible causes are rialed out:^the ^system 
used, the number of citations printed, and the interview/ 
strategy time.'' It would appear that explanations.Mjiu'st be sought 
in the personality or l$:nowledge of " the cliejitL, the interaction 
between the client and search analyst, the quality of the search 
strategy, or the topic being searched . ^ 

Relationships for the rejnainder of the process model 
ITave been confirmed. Price is affected by the number of 
citations printed,; the number of citations "printed is affected 
by Qonnect>time but not ihterview/strategy time ; and both 7 
connect-time and interview/strategy time are affected by the . 



. method o£ contact. I£ there is a - surprise here , it' is that ^. 
while the personal contact is responsible for talcing a con- 
siderable amount of' the search'' analyst vs time , the -product''ion 
of , longer bibliographies and higher prices^ it seems to bear 
no relationship to either the^..amount learned or the level of' 
.. satisfaction* - « * 

Implications - - 

There ar6 two findings: in t}Tis"study-' which §eem^ of ' 
immediate import 'to the search service manager, and^ one,^,with 
implications for future research. All three are prodjacts of 
the process model. 

First, it is apparent that any search service which 

• charges for, its product must emphasize the' se2n;ic?^ as well as 
"thfe^ search output . It is not enough to provide customers with 

. a bibliography and materials , but it must ' be done in /a- way ^ 
that makes the client realize^he is paying as much for the ^ 

' service as for the product, ''arid that he is getting his money^s 
wbrtji./ Indeed it can be^ ar^ue^d that clients only pay 'for the 

'. service, since ^hat is paid -is related^ot ^to the ultimate 
worth of the material, but to the cost "of its retrieval*, 
r . Secbnd>thp- finding that process variables Iiave no clear 

relationship to user satisfaction or the' amoimt a user "learns, 
suggests that, where necessary , economies, may be possible by 
discouraging face- to- face interviews (which take more time) . / 
and by res tricting . the length "of bibliogr'aphies. as much as 
passible. These, recommendations,, if imposed on a staff, would 
probably fail (we spe-ak from experience) . But for search . 
an^Iys^s w^o ar§)^responsible for their own budgets and^ who must 
defend ^their existence at. least in part on the basis of the r 
number of <:^±ents served, it may be encouraging to' note that 
it is probably possible to cut 'a few corners withoprt harming 
the quality of the service. \- - .* 

Finally, the need for future research into the deter*- 
minants of the^ gain in knowledge experienced^ by users as a 
result of bibliographic^ searches is apparent, given vits, -strong 
relationship with client sat is^action . Wh iie libraries and 
information centres have traditionally defined thei^r roles as ^ 
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-providers o£ informa-tion, with no guarantee that their clients, 
benefit froirr this information it appears that for search 
services, to succeed in satisfying their client's needs /'a* 
better . understanding of the * persona^l and psychological factors 
involved is needed, In< particular if it is possible to estab- 
lish in<advance the li-kelihood of a search's .success in filling 
a cli'ent's needs, then search analysts may be able to'b'etter 
distinguish between those^ clients who are likely to profit ' 
and those who are not. The failure of \}ie -^nput model invol- 
ving ..sociological background variables to explain differing 
levels of satisfaction makes it clear that whatever the vari- 
ables may, be that explain this variation, they are to be. found . 
at the individual and not |:he group level. 



/ 



Limitations 



. / The suggestions above were presented withoiat qualified-' 
tion, but it would be emphasized that they arfe mad^e within the\ 
context* of a single search service, EISO serving a special ' 
population, the prof ess ional educational community in Ontario,.. 
Canada. Given the nature of its client group, EISO's staff ^ 
primarily conducts in-de^th, retrospective s earches requested 
^ as ^a basis for re^search reports or 'background studies 1 --Xhe 

- 'length of search interviews, connect- time , and the -number ^of« 

citations printed tend to exceed norms reported in th^litera- 
ture, but are not,' we feel, out of .line with the task CWax, > 
197*6, p. 13). Nevertheless, given EISO's norms,, it may well - 
V be that: it cdn .cur corners "without damaging the 'quality ^pf 

- service V where^a^s^, anothe service could not. .Therefore, the 
•conclusions and^recommendations made here should not be con- V 

spidered ias' conclusior^ of general appLicabi l.ity. ^ 

' . ^ Conclusion " . 




The systems framework for evaluating an online biblio- 
graphic search service described in this chapter was success- 
fully applied using . separate analyses to determine the effects 
p^f^input and process variables. In the. first, analysis, a: 
number of background ^ariables measuring the slocial characterise 
tics of EISO's users were related to output vd^iables measuring 

ERIC - , > - 




client sat^s^ction. The. overall high level" of s^^fei_&ifaction 
observed, combined with the lack qf.any signifrcant relation-, 
ships between the sociological input variables and client' 
satisfaction, were -interpreted as meaning that EISO was ' 
serving all categories , of clients with equal effectiveness. 

In the second '^analys is , a number of-^e lat'ionships among ^ 
a number of process variables associated with the costs, of 
■.operation and output variables measuring the clients' gain 
Viii' knowledge and sat is-f action . - Strongest relationships were , 
between proc'es5^"v^riabl'es such as the number of citations ' / 
printed and pi(ice, between price ahd satisfaction,- an d^' between 
"the amount learned and satisfaction. 'The negative relation- . 
ship between price and satisfaction, which did not apply to 
satisfaction with the b-ibliography'.and materials themselves,- 
suggests that .services which charge must be aware that clients! 
are rating their service as well as their product.. Future 
research is needed to discover variables that dete-rinine the 
amount a client learns .f^om a search,, since this- information 
Would give the seardh analyst a better understanding of when 
arid-how a search should be cond.iict e'd , ^ ^ 
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When preliminary investigation into the creation of the Edu- 
cational Information System forj Ontario began in the early 
ye^s -of this ;decade the- world of online bibliographic search- 
ing was in' its infancy. Little documented . evidence existe'd on 
the optimal, methods of setting up a seatch service, numbers of 
users that might be expected to use it, or the -costs incurred 
in providing it. The choice of data bases was limited and 
searching techniques relaT:ively simple. Advances in the field 
have, however, taken place with sometimes startling rapidity 
and these changes combined with the . f inding.s of our own research 
have-^f f ected the development of EISO and how. it is fun today. 
The .purposes of this chapter, , therefore, are to present an over- 
view of recent trends and developments in accessing data bases 
online and to document the ^changes that have occurred in the 
EISO service . ^ 



Trends and Developments in Online Sear chiii^S;^^^ ^ 

To/' state that significant changes have taken place over"** 
the p'kst de1::ade .with /respect to 'the prodiiction^and use of 
computer-readable data bases is almost a truism. The . constant 
flow bf changes can be traced in the - ArmicaZ R&view of Jnformat'ion 
Saienae .and Technology^ youTnals such, as SpeciaZ Librccm.es y CoZZege 
and Research L-ibi^arii^es ^ Canad'van Library JoiamaZy JotcmaZ of-' Library - .. - 
Automation Jo^cr7^at. of the American Society for Iri formation^^ Science , - 
Online Reviews To keep up with the changing scene , the 'comiher- 
_cial.idata "-base .f irms^ have JD.agun^o- J^^ue-^ - news-le-tte-rs- — r 

and user manuals. Directories of available data bases have' 
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appeared and organizations associa^e^i^ith data base .gerie'ra* 
tion, ptocg]s^4rng, and use have ^sprung: up\, Thev developments / 
and trends in data bases from. 1966. througK^19 75 were docilmented 
. in a recent paper by Martha Williams X*19770 • In that paper she 
^otes -the folloving changes. * ^ 

Number and Size o£ Data Bases ■ ' - ' 

In 1968, there were 20 data bases available to the . ■ ^ 
public for information retrieval purposes. Today there are * 
over 300.. Ten years ago data bases consisted of^a few: thou- 
sand records. Now some data bases add as many as 400,000 
source references cmriudtly. 

Number'^of Searches and Users * 



In 19d5, the people in the U.S. who* used machine-readable 
' data-base services numbered about 10,000. In 1975, there were 
1,000,000 online retrospective searches performed in the U.S. 

■ 9 

and Canada excluding library automation applications such as 
OCLC. • . 

- ^ ■ ■ ' 

Subject Coverage of Dara R?.ses 

Originally da"^ ba-rjs vere developed by government or- 
\ganizations C^'ALA, ASC, NLM) , were mission-oriented, and con- 
tained scientific and technical information. . Then ^abstract ixig 
and indexing services in scie::ice and- technology began to ' v,- . 
develop data bases They were followed. by the "soft sciences, 
psychology, , education, and ^so^cial science. " Now coverdge^is' 
spreading to the arc:i of human needs', consumer-ism, chab-d care. 



Retrospective Files 

^ Because some data basje/^ organizations g*o b.^k to 1966, 
retrospective search capabilities are. expanding. 

Types and Characteristics of Source Documents . 

Far ly data' bases 'covered government reports, journal 
articles, and patent informationj and tended to be largely.-. 
b'ibliographic. 'There is. now a trend- towar^ ^numerical data 



b43es, and- ones that contain^4ess peian§n.ent ^materialr, suc^ 
3iews^ information , information a^out current resear>^* proj ects . 



Funding Sources 

There has developed a trend away from government- funding 
of mission-oriented data bases toward comjnercial organizations 
producing data bases specifically intended as searchable files 

Charging *f or D.ata Base Services 

. ; ; Early government- funded "data bases- were available, free 
to governmelit employees and contractors. There foil oWed a* 
period of subsidization of searcTh centres with little or no 
fee passed on' to the end user. This was followed by efforts 
of these centres to become self-supporting by charging for 
services. Now organizations are finding they can make a 
profit CSDC, LIS) and there is a' growing information industry.- 
This trend toward commercialism; is a direct result of techno- 
logical developments which have dramatically reduced the costs 
of data base searching. In 1965, it would have cost $1,000 . 
to search a data -base of l,OGO,pOO records. Today, it' costs " 
between $10 and $5^. ./ , - "" 

* • • . - ■ 

Users and User Organizations . ;■' 

- Thes^ haye^. shifted primarily government and 

vit^ contractors , 'to "'industry, then, universities , and now v - ' 
public libraries^. We" now have third-party users or brokers ' -/ 
of search services who s,ell "the results of their -online se.arch- 
"ing.to their^own customer^. the end-user in -such cases . is ih6- 
cus tomqr^pf ^n eitferepreneurial intermediary . 

: . ' ^ ' \ - ■ ' : 

Professional Organi^za;tions . — 
<- ■ ' . . ' ' ' ' 

Ten. years ago there were no oi:ganizations devoted to 

computer-readable data bases. In the last ten y-ears, ASIS 
has developed special . interest groups oh selective dissemina- 
tion of information (SIG/SDI) , and on user online interaction 
CSIG/UOI). In 1970 ASIDIC (-Association of Information Dis- 
Si^ination Centers) >and EUSIDIC CEuropean Society " of Informa- 
txon Disseminat?ion Centers) were chartered.. GAIS . (Canadian 
Association for Information Science) was formed in this^^ 



Publications Devoted to Data Bases : 

Whereas ten years .ago, little -was published, today over 
500 references a year dealing directly with data bases appear 
in the journal and report literature. In addition, the rele- 
vant organizations publish bulletins , newsletters , journals. 
These were followed by edited collections of papers, direc-^ 
tories, and in the past five years monographs have begun to 
appear. . " • J • 

Terminology Development 

It is startling to remember that in 196 7, the term 
"data base" was hardly, if ever^ used and that "online" was 
not known- - 

-Standards ^ 

When data bases were used exclusively by the producing 
organizations.^ ^here was no need for standards. In the inter- 
vening years, as more organizations became involved in.proc 
ducing and .process ing tapes, the need for standard! zation^ • . 
became obvious.. Thi^ area is still in its infancy. 

File Design ' 

. Early '^data bases were not ^designed for searching but 
"were production aides that came about as a byproduct of, com- 
puterxzed composition f^pi^^^'^^ardcopy publishing. Sophisticated 
techniques now allow one master file to generate multiple 
products, having differing formats and contents. , . 

'.'.■>» " . ■ ^ ■ 

. Qnline 'Search Software ^ ' - 

it is hai'd to remember that ten years ago online retrie- 
val was only a glint in -the eye of a few information scientists 
\From rather limited versions of search systems^ current soft- 
ware cont'inues to incorpiora^ -features that can accompAish a 
-variety of functions for the user . 
■ "■ 

Vocabulary for Data Base Searching* ^ 

The -trend is away . from the use of highly, structured^ 
.highly cpntrolled vocabularies for searching data bases and 



towards free-text, natural language sfearching o£ titles, key . 
works„ key-word phrases, and absti-acts. . 

Networking 

Perhaps the most important ttend ii^ the data base area 
has been spurred by the development of huge, retrospective fil< 
which are too large to be supported by use within a limited 
geographical area; -the vast increase iti the numbers and kinds; 
of -.data ba:^es and the availability erf Online search services; 
the numbers of data base users; the cooperation made possible 
by the n^ed for standardization; and technological develop-.. 
. ments in telecommijni-cations . The development of such commer- 
cial communications networks as TYMNET, Telenet , etc . , have 
in turn permitted the shared ^onlin^e use of . search systems at ' 
SGC, Lockheed, NLM, Informatics, Mead; SUNY, OCLS,etc. 

While much progress has taken place 'and many pitfalls ! 
^aye been avoided, .several problem areas sti'll remain largely 
unresplved or in the research phkse, Copyri^^ts, data rights., 
patents, export tariffs" for data bases and isybftw'are are largely 
unexplored. Document availability still poses some, difficul- ' 
ties. The deveTopment of a common command language for acces- 
sing multiple online, systems all with different command words 
anci^or ^'equence "of operations is still in an experimental 
~stagei Prcblems of cTioosing becween many data ba'ses" that 
vary considerably^ with respect to sub j ect ."coverage , source 
types, file format, record format, indexing;-.; vocabulary, etc.,. 
remain while investigation of problems concerned with the 
man/machine interface is just beginning. 

. ^ ■ ■ ' ■ ■ . " ■ * 

" The EI SO Experience . 

.-. Experience wit.h EISO confirms and emphasizes- the points - 
brought out in the literature regarding the rapidity/of.^ c^ 
The following developments are some of those that have^ occurred 
only within the- last :>^^5^and that have \af feet dd the: type of 
service provided. 




Availability of Dal;a Bases 

In the last year alone the following data bases have \ . 
become available. Language coad Language Behavio2* Abstracts ^ (LLBA) ^ 
LipT^canj and Information Science Abstracts (LISA.)', Canadian Business Pei>io- 
dicaZs Index, Canadian Newspaper* Index y and Sociological Abstracts . On- 
the drawing board and about to Tpe launched by Lockheed are NICEM 
and National Coimcil for Child Abuse and Neglect , among seven' others. 

Reloading of Data Bases ' . ^ 

. - r Newy sea^rching capabilities are cons tantly being *intro-^ 
duced . J/ie- most potentially important one °is the ability to-, 
search for words or phrases in titles or abstracts rather than 
only by desqripltors / . 

Changing S^earch Rules * • 

An example' is the shift from being able to search the 
classification codes in Psychological Abstracts by using four 
digits to^ using only two. ^ 

Changing Print Formats ' - 

Lockheed now offers s ioc different format op tions , SDC 
even more . ^ ' . . 

Coverage Decisions i/ ^ ' : 

■ ■; The mos.t obvious is^ the broader coverage in i?J^ that 
offers dissertations and ONTERIS;^tSLat continues to expand into 
the areas of curriculum guidelines, and CODE materials. 

Communication .Networks ' - 

. ^ Oxiginally using one number with Tymshafe, EJSO now 

uses. two networks CTymshare. and Telenet) , and two systems 
CSDC and Lockheed), plus ONTERIS , involving a total of. five * 
different logons. , - 

Data Base Features ^ ^ ' - 

New features ^include cited reference capability of 
Social Sciences Citation Intex (SSCI) and numerical subject codes 
m Psychological .Abstracts . . • ~ 



Ghangeis in Costs and Charging , 

These include introductions of discbunts by SDC arid 
Lockheed that are prorated according to usage; the reduction 
.of communication costs and "data basfe charges ;. and the elimina 
tion of flat fees and the" substitution of rates reflecting 
actiial costs incurred. 

Increasing Refinements to the Search Process 

^ One of the more concrete ways to grasp what this involve 
is to examine the tables of contents of 'LockheedVs Brief Gvc-ide 
to Dialog Searching and SDC ' s InfoTmart Search Service (Appendi-'ces 
E and F) , both frpra the user manuals that number hundreds of 
pages.' 




These changes have, of course, profoundly af f ected . the^ 
type of service that EISO is able to offer its users. For 
example, when the service began in March 1975, only one. data 
base (ERIC) was searched and only one system CSDC) used to 
access that file. Now several systems including SDC, Lockheed 
ONTERIS, and The New York Ti^nes are used to provide access to 
a variety.of data bases in education and the closely^ related 
social sciences. . _ , : . ^j^- 

The service has expanded its provision of original ' . 
documents by. supplying users- with a choice of . the following 
formats :. microfiche copies of fiche documents, paper copies 
of journal articles, and . paper copies of -microfiche documents . 
The. first two of /thfe-se options are- supplied directly by EISO, 
staff, the last .through an arrangement with MicroMediia, .a." 
local commercial company Csee Appendix G) . ■ ' 

Pricing policies have b"een substantially changed also . - 
Initially;^ as a result of da t. a- collected during a short trial 
period., a flat fee of $30 was charged per search. Then-, . as 
it became possible to obtain exact .comput^j^ costs incurre^by 
each search on the printout ^nd as i-t yb^ameXclear that there 
was- little link between time spent on a search by the search 
analyst Cw-ithin- the limits set by EISO) and .the satisfaction 
of the user, the decision was taken' to introduce a slicing - 
price structure-. Now the user is charged for direct, computer - 



conncect— time costs , ixunjbei: o£ dotations printed, :and a flat 
rate £or the search analyst^s time to the nearest quartern-hour 
These costs associated with connect- tiii^e and citatiogis vary' 
. with the system accessed and daHa: base used. This new pricing 
structure has the advantage o£ imposing economic discipline 
on the search analyst without reducing the. quality oiF the 
search. 

Publicity efforts have also been somewhat inodified as a- 
result, of research and practice. • It was found that articles 
and advertisements in professional journals went virtually 
unheeded and did not serve as an effective stimulant for 
users^ . Far more effective were the brochures CAppendix H) 
sent to school boards. Direct mailing to all schools in th^ 
province was tried but found to 'be exce-ssively costly for the 
number of users generated* The current methods of publicizing 
the service consist of twice- yearly (Fall and Winter) mailings 
of 12 , 000 brochures to schodl boards, faculties <>f education, 
prof essional associations , etc.; fliers distributed at Summer 
and FalL OISE student regis trat.ibris-- (Appendix I) ; inclusion 
of sessions about EISO in graduate courses 1010 : Research 
• Methods, in Educational Administration, :^1021: Retrievar of 
, Knowledge in -Educat ion, arid "2050: Dissemination and Utiliza- 
tion of Knowledge in Education. In additibn, lligh visibility 
is maintained by attendance at conferences, ^orientation tours 
for visitin-g educators, and the- packaging of bibliographies > 
*and original documents in covers with the. distinctive EISO 
logo (Appendix J) . \ ■ ■ . 

. With the start of Phase IV of " the EI:SO pro j in April\ 

1978, the main thrust of the ^Ministry of- Education contract \ 
.becime. th^e provision of the online bibliographic retrieval • \ \ 
service; Collection of research 'data^was. sharply curtailed ' f 
and all data collection ins tiruments were revised to reflect 
the shift in emphasis from research' .pro j ect . to service nfoni- . 
toring. New Search Request Forms (Appendix , -EISO^ Service 
Evaluation Data Sheets (Appendix L) , and EISO User . Evaluation 
Questionnaires (Appendix M) a^e designed to pi^ovide inf orma- 
tian on who uses the . service, the; purpose for which they us,e 
it,, their degree of satisfaction" with it, and ""t he cosjt^of 



providing it. Items referring to other^ types of information 
Jiave^ been eliminated. 

T- The role of the Education Information Consultant or gJC, . 
the ' intermediary between geographically remote users- and the. 
.centrally located information resources, that began in 197S 
is continuing though in a modified format,. Detailed case 
studies of the four part-time linkers (Auster, 1978) showed 
that these - individ.uals were indeed performing a useful £unc- • . 
tion as perceived by their users. Their greatest contributions 
tended to be in the early stages* of the. search . process in making 
the pptentdai user aware of the service and Helplng/him to 
define and clarify his problem or* information need*' .The roles 
that developed were found to differ significantly in. their con- 
figurations with the functions performed, time spent, numbers 
of users varying markedly from .EIC td'^EIC. , pne of the -unex- 
pe~cted outcomes of the- linkage program was the springing up 
of voluntary or self ^ selected EICs who had heard ^about. the^ 
service and. felt that it could be ' useful to their organization^ 
These -volunteers have, become repeat users, of - the EISO sefvice • 
and have generated numerous'' searches over the -past thre.e years, 
The role^ of EISO/.s staff members have also undergone \ . 
changes* The job of the library assistant who performs the 
clerical duties of the project has expanded with the introduc- 
tion of alternative duplication formats, individually^ tailoring 
billing systems, and regular publicity mailings^ Record keeping, 
office management, and quest ionnaire coding round out a busy ' 
routine. The proj ecf matiager'^^s functions have so-far- not been 
as severely curtailed as was anticipated at the beginning of 
EISO^s Phase IV. While it was ^thought^" that -the reduction of 
data collection and analysis would reduce the time needed to 
carry out this; role, the fact^that the role hadbeen a shared 
one for .the first" three years of the project was largely ^over- 
looked. It appears that with the responsibilities for tresearch, 
project monitoring, and publicity shifting t^ one individual , 
the time allotment for . that -role is closer to the 30-40% range 
than to the 10% actually ^allotted to it . . • ' 

The grea-test change, however, has been in the role of 
the search analyst. Her role has become far more complex. 



as a result of the technological trends and developments des- 
cribed earlier. With more systems'-ahd data bases at her dis- 
posal, s^he must constantly update her. skills and the list of 
s-kil-ls regarded as necessary has become increa'singl>t lengthy. 
'One expert . (Mar't in, 19 7 4> feels that the analyst must be know- 
ledgeable about query formulation and result manipulation. 
Thef former includes familiarity with suffice removal, search 
field control, dictionary access, spelling variations, 
related term capability, word proximity operators,: Boolean " • 
operators, request sets , phrase-, decomposition, search profiles,, 
and sequential' searching. The latter includes' familiarity with 
search review, predefined formats'-, online formatting, -rapid : 
scan,^;expanding, sorting, ranking, mi^crofiche, statistical ' ^ 
interface, offline printing, ajid data access protec4:ion. 

Another expert (Wange.r, 1977) presents her view of the ■ 
skills required in chart form^ ' 



• . TABLE 3 2 ■ ■ " • 
skills: REQUIRED TO ACCESS MULTIPLE. DATA BASES 



Skills/Knowledge Base 



Reference 

§ Infor- Data .' . . "That- 

mati-on Base - Systejm- Special 

Education .Knowledge Skills - Skill" 



o 



o 



.. AM4LY^E.. THE INFORiMA.- 
' TION' PROBLEM ' - \ , 

. SELECT DATABASE (S) 

. SELECT TERMS 

4. > CONCEPTUALIZE INPUT TO 

SYSTEM 

5. EVALUATE .PRELIMINARY 
RESULTS" ~ 

6. ' CYCLE THRU 1-5, AS 

NEEDED • 

7. MAKE FINAL PRINTING 
, DECISIONS 



' X 
X 

X 



X 



, ■•-•■X 

X 



X 
X 



8'. EVALUATE FINAL RESULTS 



ERIC 



^Wahger, Judith. "Multiple Data Base Use." , Online, 1977 
, 35 7 41.. • . / 

i35 ■ . ■ • . ^ 



15? 



1 



She -recognizes-, however, that the search process is a dynamic 
one. relying -heavily on humaa- factor s which, cannot easily or 
compLetely be captured on paper. To underline the importance 
o£ that extra dimension, she introduces "that special skill", 
which may encompass - .-. , 

* • •. the ability to relate' to people, to 

listen, to read (and, hearj between the lines, / 
to make some f airly-.- abstract ccxhnectiOnS, to ' 
feel confiden-t about entering .another person ' s. . 
.. - - • world of interest and expertise, and to begin 

relating a search problem to the world of in-- 
fprmation (p. 35). ; . ; ^ 

Needless' -to say, the errors that -can be macie by the un- 

■1 ■ ' .... - . „ 

suspectxng novice in such a sophist icat-ed; prof essional area 
are almost' 1 imitless . Ironically,, however; "the major learning 
opportunity for currently practicing search' analysts has until 
recently -been primarily, on-the-job trial' ^nd error augmented 
by professional development workshops held by the-'commercial 
(lata base brok^ers at conferences. ' . - ; V: • 

V ■/ .'■ ■- Conclusion: ; - 

V „■ -When the E ISO. pr6jec the whole field of 

opJ.ine bibliographic retrieval was relatively new and undeve- 
loped.- Few guidelines or /criteria existed for the establish- 
-ment arid-ongoing operation, of such services, co-sts were as ■ 
yet undetermined, and user acceptance was still problematic. 
Within a very short time', technological advances made possible 
the provision of a wide- selection of data bases, costs became 
more readily 'determinable , and online access became the ■ 
accepted way of ^searching ^a substantial portion of the social -, 
science literature. There can be little doubt that, 'online - 
services will continue to develop and expand to meet the in-'v 
format iojQ^-.needs of growling ^numbers of users in the future. 
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.- • APPEND IX A " 

ElSb USER EVALUATION QUESTIONNAIRE' 1 



•O 



let. 



1 






o 

ERIC 



1S5 



140 



.A 



' - . ' ' ^ " . * ■ • 

; According to our recorcU; you conductKl « SMrch through . 
th« Educational Informatibn Syst»m for Ontario (EISO) * 
during th« last two months or so. As you know, this 'tn 
cxpw-inwntal Mrvicc. and is part of a resaarch proj^ aimed 
at Tearnino mor« about the information nimfs of Ontino: 
tducators« and how thes* neods can b« mat>^ 

, CouJd w» imposa upon you to completa tha avaluation 
qua^tionnair* and rscurn it to us in tha •rtttosad salf-,^' 
addresMd and jtamped envalopa. The quastionnaire'-shbuld 
cake no mora than a few minute* to compJete. All data will 
ba heid in strict confidence and used only- for reseac;ch 
purposes. 

Manv thanks for your cooperation. , * 

Sinc^rely^ 



Ethel Austtr, Stephen'B. Lawton 
Principal Investigators . 




P.S- . If you reouestad the search on behalf of another user* 
please pass this questionnaire on to the individual 
concerned for completion* 
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CISO Umt E^luAtion QtMstibfmAlr* 



■ V 



. ^9^ coll0ctmd from th« u3«r evaluation questionnaire will^ 
be used only tor raw jrch purpo*atl To identify and attract ' 
pocenti^Jy ioterei'ted twers to El^ we murt know who J$ , 
now utrng the service; w« must also identify and Improve 
weak. area* iiT the search .servic*. 

Plaaae reed each quection carefully and indicate your 
. re^me with e check marlctQ' . • « ^ 



Search No. 



. Topic . 



If yo4i heve completed a p^evioua evaluation questionnaire 
pleaae start at item 24 on page 7, " . T ' 



7M 



t. AgeMtjn 25or-unaer 
• (2) □ 26^ 35 
(3>n 38'. 45' 
■*'<4)n 48-55 
: (5)Q 56- 65 
. ' (6) □ over 65 



1^ 



2i Highest earned degree 
.iTjO;. Bachelor's' - 

; C2) D. jyiaster's . 
OrD Ooctorate . 
(4)Q other certfficate. 




a F^««3* indicife your primary professional roTe or fun^ (Check one onlyX 



(Ot) O Administration or Supervision ^ 

. (0210 Teaching . _ , 

. (03) □ Pupir Personnel Services 

(04) Research. 

(05) Q Field Development 
.(06) D Ministry Regional Off ice - 

Library Services - 



. (DSJ O Pdvate Consultant 
.(09) O Undergraduate Student 

(10) 0 M.Ed. Student 

(11) □ Ed.D. Student 

(12) 'n w^. Student 

(13) Ph.D. Student ' - ^ 

(14) □ Qtherr 



t1 



4:. How many years have you 
J. less thari one year 

<2)0*-tyear 

<3)D -2 years 4: 
' • (4)0 3 years : . 



in your current position? 



(5) D 4 to 6 years ■ 
. (6J D.; 7 to 9 years ; 
^\(7) Q ; 10 or more years , 



ERIC 
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12.13; jts: ffav« you irv^r 
11) □ Yes 



^ . A. ■ 



/l2) □ No 



Hav* ypu^itron any prof«Ufon47vDAp«fs for 
publi^^ifOa or for presentation VtjTonferentes In 



6. Are vou e jnember of any professional orsanizatiQo . . 
joxher than those rttqutred. by provincial legislation? ^ 
(DD; Yes ' {2}.n No y?' 



8. >4ave you participated in an educational research project' % 
in the last five years? . * 



^ <1) □ Yes 



/ (2) □ No 



16 



' 9. Are^yoo.noW studying or plallTvng to- study toward > 



. an- academic degree? ^■ 



(trn Yes <2) D Not sure (3) D No 



t7.1S 



'10. If.yoo are w^th a school board, please indicate 
the level of youl' pttm^T'>/ professional*^position. 
*" Otherwise, chectc "Not applicable'*- 
' D E2cm« ntary .grades on1.y. 

(2) D • Secondary grades only 

(3) O EJementa'ry a r>d secondary 
W Central board off ice 

(5). (ILL Other: 



(6). □] Not applicable 



11. What is the mother tongue of the majority of 
individuals served by your organization? 

(1) D English " < 
r2Kn French^ 
- . (3) □ - Other.: • ' • ' ' ' " ■ " 
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r 



r 



lnf6rm»tion /Vetds . 



12. indicate (A) the amoum of tim« per week you normany spef>d in obtaining information from 

^ sources luted b«row.and (B) how valuable the information sained from each source Is to you. 



each of thie 



19.30 
23^4 

" 29.39 



. Source 

«. Human resources ; . <! * 

b. Office files, reports . 

C Prof. OTQanizaitOns /, . v. ^ . 

. <i. Libraries and their holdirigs— 
Journals, books, indexes/etc 
Conferences r 

f^ Prof. Dev. Days. . 

9* Academic, prof, courses . . . . . 

h. Currtc guides, texts. ^.V. . . '. 

' \V "Sf^ ' y materials 

J. QthX- - • 



(A) Approximate Tirne 
Spent (hl^s. per week) 



39 
40 
41 
42 
43 
44 

46 
.4T 



< 


L ■ (8} Value 


■ - 




• . Low 


Mcdiuni 


' .High 


. .NA ' 


D . 


a 


□ 


a 






u '■ 




d ■ . 


. a ... . 




o - 


■ n ■ 


□' . 


□ . 


D 


d 




n ■ 




■ ■□ ■ ■ 


□ ■ 


D ■■ 




□ . ■ 




. O ■- 


CI . . 




D ': 




□ ■ 




o ■ 


D ■• 


□ 








: Q ". 


(1) 


(2) . . 


(3) 


Ml 



r 



a. How often* do 'you seek m formation for the fQiJowing? 
Purpose ;,\ - . : -J_ f 

a. To keep abreas^in tfSw" field. . : ... . , 

b. Assignments/ cerirn t>^^T^^ or theses^ . , . . 

c Preparing or.uojJating b^ibriographies . . , . * , . 

d. Curriculum develbpmept . ; , . . .... . 1 ^ . . . : 

e. Program" improvement . > . . . ; . . ^ . . .\ 

f. Preparation of speeches, reooftsl articles: : . . ; . 

g. Research and devetopment proieccs , r^'TT^.v, I'i i . .' I 
h' - Browsing '.\ : i * ^ . ."..,,1 J . . V*. . 
i.; Personnel fecruitirjenr or evaluation. . . . \ \ : 

• j: Policy development . i . . . ; „ . ^; . . '.^ . . , : . . ^ . ^ 







1 

Prequency - . 




Never 


Sometimes Olren 


Very often 


:d ■ 


u 


. - o • 


. a . 




■"■ a 


n . ■ 




D 


■' .□ V 






.□ . . • 






. D - 


□ - " 


■n ■ 


. ' ' d- 


. ■■ n ■ ■ ■ 








a 


■.■' 




□ '■'■-< 


. ,D - 


; -□ ■ ■ 


- a.' 




, -' d .- 








■d 


□ 






a ~ 






□ 


n - 


- <t) - 






(4) T 


■ ■. ' 









/ 



/ 



1-4 4- 



1^ 



SiSO Smsrch infprmm^on 



t4. 



S5 



5G 



How did you.firsc J9«rn about tha Educational 
Information System for Ontarro (HtSO) and its 
•ervic«7 (Check 6n« only) * • > 

COD O; Brochure or flyer 

Adym^'Sement •/ ' ' • ' ' (■ 

Prof. Dev. Day - 
Oassroom instruction ^ 
ColIeaguW . 



(02) n 

(03) □ 

(04) □ 

(05) □ 



tS. Is a microl»ch» reader. available in your organFzation? 
•(2)'n ^Oon'tknow , • 

' - arn No - • ^ 



1 



. - .'^'^ indicate yo*^r latisfactipn with the'fdllowine- 
. eJenwi^of EISO. Feel, frw to' comment In the space 
provided. • - V* • . ' 

^ Conveniencff,of arrangements and^adecniady of . • ' 
. directions TO obuiii EiSO searches" , * 
•".Commeoti; ' • - • " . -^^ 



18. Accuracy and co^prehensivenei of pOb*i 



materials, ■■/':.[ 
. Commenti; - - 



City 



79.- Adequacy of directionj foi: ordecing copiei 'of 
, - materirfls.Iisted in bibliography.' / 
-Comments: " '* ■ . ~ 



(OS). 

(07) □ 

(08) □ 

(09) a 
□r □ 

/ - ■ . 



Ed. Information Consultant 
Library or Librartart 
EISO staff member 
Cannot recall 
Other: 



16L Portable f iche^ader? 
r T[i)n Yes 

(2) □ Don't know 

.'. (3) □ No 



Satisfaction ' 



Low 



Medium 



High 



a ? ' 
(J) • 



(2) 



y 
7 
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57jsa 



20.^^ B&fo^0 uung BlSO, how h«ipful did you Mp^ct it to be? 
(T) n Not Wy tielpfuf 
<2j . □ SomcwHtt h«ipfij| 



89.60 



22. Would you us« ElSO again? 



(1) □ 

«) O- 

(3) □ 



Yts 

Ptfhipc 

No 



21. 



Would you racbfnmvnd usa of &ISO to a tioUaa^ua? 

(1) □ Y«s " . ; 

(2) : □ Perhaps ' - 

{3J □ No^ : ■ ■ " ' . ' * 



tfJt3. Do you think ElSO saarch ftrvices" be offerad on a 
permahanc basis? ^ 



(2) D 
tt) □ 



Vas 

Perhaps 

No* 



Thoaa who hava compfvtad pravioua quattionnairaa. pfaasa start at tfib poictL 
Othan, p laa i a oontjntja. 



24. Pteasa indicate tha profaisjonal%la for which Voo > 
. «^ wasted this ElSO^lshography. For example, -if the- ' 
bibliography was used to complete a icqursa papar fcf - 
your you woufd check t-M.Ed/sttidifir^ aJthoush 

. . yourpfimary prof esiibnal rola or function is m 
. administration. {Chack one only) 

(Oil Q Adminiso^tiori or Supervision 

(02) □ Teaching 

(03) □ Pupil Personnel Servtcas 

. (04) O Resa^rch • 

(05) □ Field Development . ^ 

(06) □ Ministry Regional Office 

(07) □ Library Services 



(08) n PHvate Consultant 
iOS) Q^Undergraduata^Studant 

(10) □ M.Ed. Student 

(11) n Ed.O. Student 

(12) D MA. Student 

(13) □ Ph.D. Student. . 

(14) □ Other: 



ERIC 



146 



25, How mudh^id you Uam about your topic as a 
r«ult of thii March?. 

:n> D Nothing or very littia 
V (21 □ Som« 

(3) □ ' A gmt deal 



26. 



6a 



TO 
73 

7tt 



27. How much additional information do you still dasJr* 
concwTiJng the topic that was saarchad? 
/ ' it) Q Nothtng or vafy Jittla' : - ■ 

^ (2) □ /Soma \ . • 

. ; a) a Aora« dia» : 



2S. • How haipful wis tha information providad as a result of 
. ^ your ErSO saarch for aach of thv following activitias? 



a. Kaaipang abrwit of the fiald \ . . . . . . . . , . 

b. ^ Completing assignmants, thai«^ mtc^\ , . , 

c. ' Ptaparing or updating a bibliography^ ... , 

d. Curricururadcvalopm«nt 

a. Improving programs * . . : *, . . . < . ^V. , . . , / 

. f- Pt^paring a* speech, articla or report. .... 

fl- Mndaftak.ing.or complecin^Vrtfsaarch or. 
davalopment project V. . . ; 

h. Browsing _ . 

i. Recruiting and/or evaluating pienono^r. 
j. Developing policy. . . . J . . . 

k. Qthar;' .- 



How much did you learn about the Educational iWr* | 
mation Sy^em for Ontario as a result of your search? 

(1)-^ Nothing or very little ^; . 
(2rn 5ome' 
(3) □ A great deal 



Helpful 



Low 


M«dium . 


High . 


HA 


a 


a- 


□ - - .V 


a 


o ■ 






a. 




o- • . . ■ 




a 


a 


■ a . . 


D. 


n 






n 


D' 




□ . - 


o; - . 


D 


□ 


o 


□ 


D 


D > 




□ . 


D 


GT. 


■ ■; . n - ■ \ ■ 


-D . 


n 


□ - 


D .. 


: □ - ' 


a 






:0 


D- 


ny ■ 


. (2) 


\{3) 
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-14 7 



V 



% . ■ - 



search No; F r f >^ f 
(1) 



29. For hew many of th» rm U tmi ca s on your printout dtd 
you r#»d th« ori^'nal m»t»rial? - 

(t) □ 0.5 . . 

(2) a 6-tO . T . 

p) n 11-20 

W D .21 ormoftt 



Commants;. 



•.9 



3a How much of th# r«i«vant information rotri^ved wa^ ^ ^ 



n«wtoyou7 
(t) □ 0% 

(2) n 10^20%. " 

P) n 21^50% 

W Q /mora than 50% t> 



3^. How do you faf I about tha praca of tha taarrfj>*- 

/N (1) □ Vafymsonabia * 

<2) O Acceptabla :v 
- -Pi □ ExdKsiva ' *— ^ " 





Plaasa indicata your satisfaction witft tha.followT^ 

alamant* of your ElSO saarch. Faal froa' to commant 
»n tha spaca providad balow. ■ 

:- ' ■. ■ ■ • ■* 

. 32?H«lpfulnaM of saarch anaJyst'or Educational 
Information Con^ltant. 
ComnMntY* " 




Satisf«ct>an 






TO 


''^ -.Low 

□ 


Uedium 

•S . ■ ' 

a. 


High' 

d 


'r u 














Tl 


32- Langch of bibliograp#>y 




■ 0" 








M ' 












33. Haadabtlity of microf icha'copias. 

Comments: . 


□ 




□ 


' a 


















(1) 




P) 


(4) . 



■ ■ *■ ^ 



14 8 



Fj 




t3 



14 



FImm indicate your sati^f actron with thm f orJowing ' 
•lemtntt of yoOr EISO search. to cdmmtxx 

in th« spac0 provided b«Iow. 



34. Tim» takan to deriv«r^tbe EISO,biWiog-4phy. 
Commants: ■ » . 



13 



37 



38. Tim. takan to dali^ microfiche or paper copt« 
. ordered from EISO. 
Comments: ■ ' " 



385Value of tsibliograohy; itself.. . ^ 
- . Comments: ■ 



Low. 



Satisfaction 



Medium 



High 



37. Time taken to deliver microfiche or paper copte? 
ordered from EORS: 
Comments:^. : 



aS/ Valueofmate^-ijH located via ElSO bibliosraphy. 
Comm^m^' ~ • . . 



40. Availability-of microfiche readers.. 
Comments: • - 



J. 

a- 



NA 

o 



(2) 



14} 
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of your EISO Mrch. Any t«9o«tioW* yoi* m^k* 
^ towardi. improving th« 4«rvic» m w«lcom«; 



T'^^'^^K you for your time and coopers tsoa. 
>Vm5» return completed! quf^tionriaire xo: 

EISO Search Analyst ' - - C " ; 
OISE Library: S218 ' • " 

eioor Street Wes^ ; " . , 
Toronto, Ontario M5S TVS . ' 
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■1 f^rr-- 



J' 



. APPENDIX. B 
EI SO SEARCH. REQUEST FORM 1 



■\ 



ERXC^ 



EDUCATIONAL INf-ORMATIO>/%Y5TEM FOR ONTARIO. 

Th» Ontario Tn«itut« for Stucltes in Education 

Room S218. 252 B5oor Street W.. Toronta>JW5S 1V6 = 
(416)923r-664VExt.487 ^ !' 




Phone I ) 
3. Bibliography to 



Search No.. J 
Topic: 



2; - lnvoic«r ( ) Sam 



or 



Phon© X J_ 



4. Search Topic::^ ' Ywdy to be searched 
Detailed descnptson of topic ^ " 



Level or grODp . 



R=ajon, fo? re^ue«in3 s«»rch {pr«,5e jpeclfy how you hope to use theJnformation} - 



Number of references desireS: 



CtatTons of Relevant materia!, known to you. if any", or authors writing in ^he 



^ area. 



Preen. 
In person- Q 
Date request received _i _ i 




Length of interview . 



\ Telephone □ M^r D ^ ^ 

-L . Date of intervfew i i y 



M 



_ min. 



Does ihe-client have access tq microfiche reader? 
ElSOTleader?' Yes. Q . . No D 

Search Strategy — 
Comments:' ^ 



Yes D No D- ■; ^ Donftknow D 

Don't Know □ * 



/ 

- / 




. A-PPENDIX C .... ■ . . I 'oK 

E I SO:. SERVI CE EV:A^'l>AT ION. DATA SHEET 




<3> 
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17 



25-26 



27 



4Z 



43 



44.':> Run , 



avelopraenl- repior^' 



Search RegOesi" 

42.. Rc'd. ....... ^ 43. Intvw. ....... 

■■^•>-, ■ ■ ~ : - . ■ ■: . 

.45. Purpose C01)_ Keep abreast of field ' - C07) R e^ 

- , (02) Assig^^menty +erm pepei*-, thesis' ^ COS) 'B ng- g . » • 

J*rep., update of bibliography (09^ P e. r'*:^^ ^ recruit, eval. , sup. 

- ' C04) C urriculum development . CIO) P olicy developnTghi'* ■ : 

iOSy Program improvement C ID O thers 

(06) Prep, speech, ar+lcio, report . 



46. Contact (1).;^ In person" \ (2)i Ph<3Ae 

47. Earliest yr. rqVd. 



(3) Flali 



28 



48. No. ref. rcf 'd , K' ^ , 49. Ref exp/ ; , , / , 

■ ■ 30 . ■ < 34 - 



5(3. Cit^ known 



51 . . Price 



40 



■V*. 



(2) Invoice % C3) C ost* recovery 



49 
50 



52, Payment (1) Cash or cheque^ 

Charge for search . \ ^ - . ^ ^ ' ' ; 

53. V/ho pays (1) ^ E l SO Ci^re?) (2) R equestor C3': prgan/'' 

'V ■ (4)^ Other" 1 ' 1 

."54! biSE chsrge Unit . t \ . 

. * ' ^44 ■ - . * . ■■ . ■ ■ 



Search 
. 55- 1 ntvw. , strat- t If 



46 



56. System* (1) _>s6g ■ - t30 L ockheed 



57V Dat^ base (1) ERIC (2) >sych. Abs. C3 ^ '-^i^ ScV T (4 ) .-E CEX ^ 



C5)_Diss, Abs. (6) <ERIC + P 



(9) Other: 



58. Connect time ^ , ./m ln^ 59, 
I T • ~ 51 - ^ . 




EfitrC"^ +"^ECEA', Ca) ECEA PA 



nted >; ^; ^ , , 60. Cost/ci-t. , - ^ 



54 



.58 - 



- V 



-ERIC 



15 4' 

17- 



/iJ:. 



1 



61 



71 
72 



c: 

4 



31 



(2) 



6U Bib. sent CI) pequestor <2) Search Anal yst. <3) EIC (4) Oth^r: 



62. Date bib. rc'd. , > . ^ . ^ ♦ 
* ' '62 



63'- Cost/hr -system / , w^- » . 

68 . 



Material Charge ^ 

64^.^^Paynient ( 1 ) 'C ash or cheque C2) _lnvotce 

.65.* Who pays (1) €l$0 (2) Requestor (3) Org. <4) Other; 

66, 0 1 SE Charge U rr \ t > ^ , 

^ 73 • - • ' ^ ■ . . ■ ■ 



68. N El SO art'x:! ordered > , 



Materials : 

67. N EiSO fi Che ordered ^ ^ . ♦ 

' 7 ■ 

69. I^EISO f iche orders filled by OlSE , , , , " - 

■ • ■ * ' .. ' : ■ T3^ * ' . ' : 

70. N EISO f icho orders f i I iW by other.! Ibrarles , ^ \ ■ 

-■: \6 

71. N EISO f^che orders untl Med' . . > . . 
; J "19 

72; N E ISO art»^l orders f H I ed by OfSE , V , 
. I-^" ■■* ^ . ^ - ■ ..' -22 , . 

73. N EISO art'cl orders filled by other libraries , , / 

' ^ ' ' '■ " - '25 

74> N EISO art'cl orders unfilled ^ , ^ ^ 

28 ■ . 

75. EISO reader avMbI CI) . Yes (2) Dan't^iknow C3) No 



10 



76. F^rTce f irst search , 



32 



34-35 



77. 

r 



Please, indicate the type of organization* or Jnstitution with which user 

Ts priniarily associated by placing a check tv^) in the appropriate space below. 

(01 > preschool ' 

<02) public board - ' 

(03 ) . -S eparate - board^ ' '~ ■ * * - - . ' 

(04) ^* priirate— school or board' - . * ■ ■ 

(05) _ CAAt"- . - ' ... ' / 

(06) _^ Facul ty. of ,Ed. or Teacherp^ Col lege ^- 

.C07) • U nlversitv ■ . 



r 



ERIC 



■I'ss 



/ 



(OS) Ministry of EducaTicin. / 

t09) P rof esaftocra I Organ Szat ton 
^ < 10.) G overnment (rJot Ministry of Education) * ^ : 

( 1 1 ) OusJ'ness ;or I ridustry : . . 

< 12) piSE Di rector »s Of flee/ Ass Is tant^ Olrector^-'s Office 

(13) - F inance- _ . 

04) " C omputifiq Service ' . 

C'15) P ub3 t cations^ -.'^ ' . • - 

X:3 6 ) L I brary ' ■ *" 

<1"7^ R & ' . " ' ^ 

( 18) E ducationa I Administration " '. 

<19> Appi led Psychology 

(20) / * A duTt Educart-ion . . * " - 

(27 > ^ H istory & Philosophy 

(22) "'C omputer Appi i cations/Measurement^ & Evaluation \ 

(23) G urrlcu I um/Modern 'Language: Centre ■" . * 

(24) E ducatTonai Planning 

(25 ) S pec? al Education 

C26) S ociQloqy in Education w 

(27) Graduate^ S^tudies 

<28) F ield Qev^lopment 
(29) Other: A 



S 



4 



78. Is the orgfani zatTon locate'd i n 

. "COn pr;strlct or Distj-Tct Municipality 

t02)_ County or Reqional Municipal liV 

(03) M etro Toronto' 
<04) O ttawa 
(05) London 



(06 y H ^7 I ton 
(07)^ _WilhJjspr 
<08) Sudl^urv 

(09) Canada (outside Ontario) 

(10) O utside C;9ffia<ia , / 



79. Is the organization located in the 
Northeastern Region (Region 3) of 
-the Province of Ontario? 



(1) 



Yes 



(2) No 



80. 



A: 




What IS the corwunlty the 
organ Izatlon. serves? 

(1) _Mostly rural 

(2) Eq.ua 1 ly rural .ancj urban 

(3) , M ostly arban 



40 



^1 




.<4 3 



'•81« Who-^osff^ned the. search- request? 
Cn ' S earch Analyst 

<2) pISE Field Centre EIC 

^3)__^FacuIty of Education Etc 
' ' MiTiistry pf. Education EIC 

(5) ' CAAT g>C 

C6) N ot sure * . ^. 

C7) Other: • > 



82. Sex CI) ^ Male 



C2)_ Tenale 



V 



\ 





A 





^ ■ APPENDIX^D 

r ' ^ : 

INTERVIEW SCHEDULE 



4>- 



0 

; ERIC 



Educational I nformat ion System for Ontario* 



I ntervTQw'^ Schedu I e 
(Revised 1977 09 19) 



• 1 fo vetri f y queist ionna"? i^e data^^ 



^fThe ijlipterviews. wi^th JElj^tcl^ primary purpc^es 

2* -to probe more deep ly ' than the questionnaires Into 

a) sati sf action'of clients, 

b) .uti I izatioa of. infonnation r^trieved^^^ t^ 

c) effects of use on^: c 
, r) 'seconil usenp of the intormation - 

^ i r): A recommendations to others re use of seryicQ. 

PART I r VERIFICATION OF QUESTIONNAIRE IXa^S ' 

^ Interviewer Tape following i nforma^tion first. 

Search number and^topic. (For multiple searches, refer 
las:t search). ' \ ^ 




to 



Name: 



Top I c: 



Number 



2. Address and phone: 
Address: 


'. : > — 




■ ■ ^ 




Postal Code: 


Phone: Bus. 


Res. 





3v Type of Orgarrr zat ton with which prinari I.y as?ocia+ed: 



4.. Rol-e in which search requested i ^ 



5. Could you tel l me about ^Hag, context search; e.g., 
■■. .the circumstances that prgmpted it. : 



' »Funded under contract by 'the Mini s^t'ry of Education, Ontario 



ERIC 



3:59, ; 



.J Jj . : 

!• -^Satisfaction (old items 45, 46; new items -3^,-^, '29. 30) • 

w^-t^ qtiestionjialre you re;^^^ * ^ 

\ V hlQh) ''level of sat f^actitDn with ya^! ue^T f he^bTf^ 

: \^^Jt^e Cou I d you ^pand bn^ th ^s;"ej«) I al ni rig why you^el^ 
^ ^ :'^his way? ' .Z/,-''-: * - • '-^^'^V.' - 

. |Samp I e probes: Was thei' format satisfactory? Did you under- 
^sc:^sta^ it? V/ere the citations relevant? Be specific — if not^ 
^ - why not? Did the search analyst understand the questionv^ 
V:^' properly? If not: evidence? Wrong terms used? Did you ^• 
contact her again tp let her know? ^ "^^^ 



■-[^J^yy}^ level of satisfaction with 

■ i ia I ocated v i a thfe b lb I io^raphy Cou I d you exp I a t n 

^ how' you cane to th^sf concl^ 

Sample probes: What dixj you do a bojff getting trj^ 
order them ;f rom El SO or go to the ^ bra ry? How mi^ you 
; read?^^^ you judge them agarnst (e.g. book^A pr 

knowledge)? . 

c> V/hat suggestions, di> you have that might impirove the qualit^y^ % 
' '''^ df EISO services and product? ' v 

Sample probes: More information on" certain topics; books; 
curriculum guides; Canadian materials; etc. 

2* ' Ut i I I 2at ion of ' i nf ormation^ retri eved. Cold- item 32; new 28) . 

Did you use the bibliography?. If so, how? V/hat was the final 
result (research report; new program; whatever ) ? Is a copy - 
avai lab le (that we can have)? What contribution did the EISO 
search make to the overa I I project? to the "backgrouad r:esearch 
phase? . ■ " ' ' . 

b) Did you encounter any problems in using it? If so, what kind 
were they? Did they relate to the service or to local 
circumstances? What would have/^helped ypu resolve this problem? 

. c) What became of the bi b I iography? Is it In your files? 

d) Have you used the bibliography .again for another purpose (e.g. • 
. class materia Is)? If so, is there a ''product" aval lable? 

Can we get a copy? ^ . ^ - 



3.- Secondary Uses ;; 



a)' D'Ld you pass/on the bib i iography , or a copy to anyone 

else? Who? "HBe specif icV^-sp we can contact the^.) V/tiy? 
bid they give"* any feedback bn^^it?! 



b) Did you recommend use of the El SO service to anyf cofleagues2 : ' 
- <o Id-i+em spy newT2 1 " ; Who?" ' Vmy ? I^rd 



on it later? 

,c) .^Can you think of ^ny other "secondary" effects of your use ■ 
■ - , E I SO, or of the b i b.1 i ography? For examp I e, did you pass 

V -'i :<>n copies of; articles; etc ' ■ 

": '-J " -v ' ■ . • . * ' ■ - ■ ^' • 

.PART I !:l : QUEST I ONS RA I SEO I N REV I EW OF QUESTS ONNA I RE - ^SPEC I ALtY 
FOLL-OW-UP OF COMMENTS ^>?V.pe 'br', write these in^ rrst. ) - 



I 







* ■ 
















V ■ ■ 

























- « ■ ■ • 



■y 



-CLOSING: 



Thank you very much: for yoar interest. I .c^.assure yqu thai- 

all th^ i^Jformatibn is conf idential ; your name and- pds^ltron wi-l L not 
appear In any report. 'Finally, I ^d like to leave with you a. copy of ' 
the guide to EISO, Search Analyst's' card, and several copies of our 
brochure. ' . • ". ■ * - . 
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LOCKHEED DIALOG® RETFflEVAL SERVICE 

^ - • ~ ^ PSYCHOLOGICAL ABSTRAfCTS^. : ^^--^^^^ 



FILE DESCRIPTION . ^ 

PSyCHOLQGICAL ABSTRACTS cover* the world's lifertjhure in psychology and relabel drsciplines i# 
th« behavioral sciences* Over 900 periodica fs and 1500 books; fechnicaf reports^ and moncgrophs 
'ore sccrtned eoch yeor fo provide coverage o^ orTgindl research^ reviews^ discussions^, theory, con- 
ference repor^5, poneJ discussions, cose $tud1^> and descriptions of apporatus. 



SUBJECT COVERAGE 

The following generol fields are covered: 

• Applied Psychology 

• Cognitive Processes 

^ Cornmunication ond Longucge 

• Cultural and Social InflwerK^ea 

end Behavior ' " 

• Oevelopmentdl Psychology 

• Educotiono} Psychology 

* • Neurology and Physiology 

SOURCES 



PerceptucI and Motor Performance 
Personality ^ 
Professionals in Psychology 
Psychometrics in Psychology . 
Treatment end Preverition 
Physical ond Psychological Disorders 
Animal Psychology 



Each year the APA staff examines approximately 2^400 primary sources, both domestic-end foreign, 
including th^ folio wingi ' ^ ^ . , ' 

• Periodicals - ' . - ^ 

• Technical Reports ^ . " 
Monographs ' 

• Books' ^ ' . . • ■ ■ - 



DIALOG FILE DATA 

Inclusive Dates; 

Update Fr^uency: " ' :^ 

File Size: 



A: 



1967 to the present ^ 

Monthly (approximately 2,000 records each month) 

240,000 citations, as of September 1976 \ 



ORIGIN 

PSYCHOLOGICAL ABSTRACTS is: produced by: 

■ -.'^ • - . 

. -s - Americari Rychological Association ^ 

1200 Seventeenth Stree', N.W. 

. ' Woshington,. D.C* 20036; — - 



Telephone: 

- 20V333-76C0 



PSYCHOLOGICAL ABSTRACTS 



SAMPLE RECORD 



YR=- 



AU=- 




1X>C VHAPt 1 97fi VOL W)i 55 ABSTRACT NOt 10895 ^ " _ 

"<CKHflcat!on of agnress^lvo behavior In two kindergarten chl Idren. ^ 
'*!/3s»r, R,« FloscJorf, P. » Ll'nbouro, Marine _ , • ' ^ 

U Tt^Incen, i5syciiolo<jlsches Inst. H Cer ni^ ny ^ 

-Psychologic In Hrzf^rfiung un<i Unterrlcht 19^4 Vol 21 C-*) 24<?-253 
^elected -3 aoQressdve boys from 33 klnrferoartners an rated their 
i»ogres^ Iveness for 6 hrs .on II days* For the nert II days 
«gnres^lv^e?^s was followed bv a removal from the ro.ot»i» the Tencth of 
'absence chosen by the child after the 'reasons for t>iB Isolation had 
heen explained. After IP wks of no trea^tinent* both boys were observed 
for another 11 days* The less, aggressive boy had abandoned his 
stronoly aooresslve behaviors^ the" other one had resiimed them» thoooh 
not at their orevlous hlcjh lovel, 

ATiCXJACB- Germ CLA^??r gTrAxrf>*f- . , . 

sudJS?C'i refe.u<- A&3ftK5(vi? ^cHaT'Ioq^ CLASSwn** behavior modification,^ 

ICTNiJeRii^ARTEN 5rru ng»rr<^^i -^on _ 09AIO» ?7370 y 

( hO^X, PriPASc- Tpnoth of remove I from c la ssroom fo 1 1 ow ino - 
anQress Iveness, aogresslve behavior* Icindernarten njalos 



RETRIEVAL METHODS 



SUaiECT OR TEXT SEARCHING 


SUFFIX 




EXAMPLES 


Nonm 

AI - 

/DE 
/lO 
/CS 
/A3 


Scacc In<i«x 
(Tttl«, gUcripM, 
^lavttiftT, Corporate 

^ Source^ AbsJrocf) V 

f;Me 

Dsscriphar 
IdenHfTer 
Cbrporcfe Source 


E AGGRESSIVE S CHILD? 

S AGGRESS*VE(QBEHA\^OR(^MQJ31&CAT10N ^ 
* S TUBTNGEN(F)GERMANY ' 

S ICIN0ERGARTEN(W)CHILDRENAI 

S CLASSR0OM(W)BEHAV10FV^DE 

S CLASSROOMCnREMOVAl/ID 

S PSYCHOLOG1SCHESOV)INST/CS . v 

S IS01AT10N(F)TR£ATMENT/AB - * 



CODE SEARCHING 



PREFIX 



FIELD KAME 



EXAMPLES 



.AU- 
CL=- 
CC^ 
JA» 
JN=» 

LA- 
5C- 
YR- 



Authoc 

Classification Codm' 
COOEN^ 

Joumol Announcemenf 
Journal Name 

Longtiogo 
k^ubject Code 
Yocr of PublicoHon 
in PSYCH ABS 
Updo re 



SELECT AU*Buuer, R- 
SELECT CL=»35 
SELECT COPSYC-A 
SELECT JA»5505 



EXPAND AU-Flet^dorf 
EXPAND CL=<50 
EXPAND CO«PSY 
EXPAND JA*S500 

EXPAND JN'PSYCHO- SElECT JN-PSYCHOLOGIE 

LOGIE ERZIEHUNG, UND UNTERRICHT 

EXPAND LA=FRGN SELECT LA=»GERM 

EXPAND SC«013 SELECT SC=01390 

EXPAND YR=»70 SELECT YR-76 

EXPAND UO-7600 SELECT dD-7605 



^CODEN expandable & selectabU bur is not displayed wKeo ct^atIoft T» prlnfed. 
The LIMIT comtpond xs not appKcoble In File 11- 

FORMATS AVAILABLE - y'^ 



Formo^ 1 
Former 2 
Formor 4 
Formaf 5 
Formof 6 
Forma f 7 



DIALOG Accession Numbers 
PA Record wlrhoiJt^^Vbsrroct 
PA Ciration end Abstrccr 
Complate Racord . 
PA CitaHoa Numb^r'and TlrU 
PA -Ci ^ction 
Abshtscf 



, Source. Docunwnf OfcMoo^ or»d 



SORTABIIITY 



Seoreh rewltj con be reoncnged for oFH me prTnHng by using the ^sorting copobnify 
wath the oFHine PRINT commcod. Sortable fields ore Aufhor and Title. Ascending order (A-Z) 
win be used unless descending (Z-A) IS specified- 



.EXAMPLESr PRINT V51-94/AUAI 

PRINT 7/^/1 -2Q/AU, OAl . P 
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EduMtional Informition 
Syitam for Ontario 
OlSE Library. S 21 8 
252 Bloor Street West 
Toronto M5S1V6 
(416) 923.6641 ext. 487 

Order^Formfor' 
Microfiche Copies 
of Documents 



A. 

\ * 



SHIP TO: 
Name 



Address, 



Phone ( ). 
Data 

Search Nol 

ERJC. 



InstTuctfoni ^ 
I Enter ED numbers in columns below , 

biXraXV V °^ ''"^ 

3, Retain 3rd (pink) copy of Order Fbrmjfor your ricqrds 

4, Mail remaining copies«to EISO ' . 

5, When we reply stating cost of order send cheque or money order payable to EISO 
The microttcha copies will then be sent to you. , • 
EDNumberi EDNumber. \' ED Numbers 

7 



n 



£1^ 



A 



2np 



1) EfSO 






Order Form for 
Paper Copy pf 
ERIC Docuijients 

ED000001-. 
ED 065 734 ' 

from EDRS 



ERIC Document Reproduction Service 
P.O. Box 190, Arlirtgton, VA" 22210 
Phone (703) 841-1212 

SHIP TO: 



/ Name 



V- 



• Addresj 



V 



.Phone ( ) 



SEND VIA: 

□ Parcel Post □ Air Mail 
Date - 



ED Number 



- No.of-Paget 



Cost 

{(^fficeu5»«nty) 



7 



Total 



' Enclose payment in U.S. funds 
. ■ payable through U.S. bank or 
. ' clearing house (e.g., bank draft) 



SPottagt 



STotil 



DEDRS 



Educatipta! Information ' 

System for Ontario * 
i OISE' Library. S 21 8 

552 Bloor'Street West . ' ^ ^ 

Toronto IW5S.:V6 
. (416) 923^6641 ext.W . 



7 




Form for 
Paper Copy of 
ERIC Documents 

ED 065 735 and over 

fromEISO . 



For each document desired, enter ED tiumber and number of pages in 
first tv/o columns on following page. Leave third column blank, 

2' ' ' 

Complete address section of form (search number is given at top right 
comer of search reijuest form and also on first page of printout). 

Retain 5th (pink) copy^ fonn for your recorts. 

4 

Mail remaining copies to EISO. ' ■ 

* ' ■ 

. When you receive documents and invoice, send cheque or money order 
to EISO. > 



Si. 
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ED Number 



Educational Information 
iSystem for Ontario 
OlSE Library. S218 
252 Bloor Street West 
'Toronto M5S1V6 • ' 
(416) 923-6641 ext.487 

Order Form for 
Paper Copy of 
ERIG Documents 

ED 065 735 and over 

fromEISO 

SHIPTO: 
Name . 



Address , 



Phone ( ) 



SEf^DVIA: 

□ Parcel Post 

□ . First aass Mail 

□ Delivery Service' 

ERIC 



EISQ Search No, V r 

1^ ^ 



Order No. 



(office use only) 



'A 



r No. of Pages 



Cost ' 
(office use 



Total 



205 



SPostasi 
or 
DdlvBry 



$Out 

EISO 



1) Requestor 
Imroica 





•^dueationaf Information 
Syttem for Ontario 
OiSE Library. S218 
252 Bloor Stfeet West 

■ Toronto M5S 1V6 
(416)923-6641 ext487 



Order Form for 
Photocopies of 
Journal Articies 



ln^struction& 



1 

For each Article desired, enter journal informatiqn on the grid on the 
following page. Samples below show where infdmiation required is listsd 
on the printout of your bibliography and'hbw it should be listed on the 
order form. ■ 

Sample Printout item: ^ ^ 



EJ1371QP TM502226 

Creativity and Body Ima^ Boundaries / . 

Loshak, Lee l;fli2nikoff, Marvin Journal of Personality ' 
Assesanent; 40; 1; 81-90 Feb 76. 

^Descriptors:. •Creativity/ •Body ^ Image/ •Individual 
Characteristics/ " •Creativity Tests/ Higher Education/ Males/ 
College Students . 

Studies creativity as it relates to boily image, and examiri<«j 
the communality of several creativity measures. (RC) \.'^ • 



■ » 



, VoL ' Issuer* 

Journal flame . r, No. No: 'feotdl Dirt. 


JoTunal of Personality ^ 
Assessment" v ' yi^ 


40 




81-90 


Feb 76 























2 



Complete addr^ section of form on the following page (search number 
if given at top. right corner of pink copy of Search Request Form and 
also on the first page of the printout of your bibliography). 



Retain 3rd (pink) copy of Order Form for your records. 
Mail remaining copies to EISO. . " 

■5'' ' 

When we reply stating cost of order, send cheque or money order payabl^\. 
^toEISO.Thephotocopies.willthenbesenttoyou. • '^^ 




Journal Name 



Vot. ttsue 




EdutfOoiul Information • ■ 
ST^ttwn for Ontario 
OlSE-Librarv^.Sil8 - : 
252 Bloor Street West 
Toronto M5S 1V6 
(416) 923^1 exL-487 

Order Form for 
Photocopies of 
jQumal^rticles 



SHIP TO: 

Name. 

Address 



Phone { ). 

Date__ 

Seardi No. 



ERIC 



• 




_ - 














■ ^ 


r ' * ■ ' ■ ' 














- 




.> a 






V 




• 




















\ 










\ 










• 
















■:/' 






















• 
































• 


1 

— — ~ 






























: ^ ' 






s , 










\ ^1 
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• APPENDIX H » 
EI SO BROCHURE 



ERIC 




t 



Educaiional 
Information 
Systen\ 





EISO Search Analyst 
OlSE library, S218- 
. 252 Bloor Street West 
, ' Toronto, Ontario M5S1V6 



Educational 
Infbrrfiatjon 
System 



i t 



iHaiie you beard about EISO?. 
Chances are you face problems i)ow where additional 
' information would be of help. If so, the Educational 
Information System for Ontario (EISO) project js 
ready to assist by, locking and duplicating materials 
contained in the collection of the Educational ' 
Resources Information Ccr)ter/|£R]C), and other data 

■ bases of interest to educators/ * 

What is ERIC? ' ' 

■ ERIC is an information gathering and disserffinating 
network comprised of ISsubject-speci^ised clearing- 
houses operating under the auspices of the U.S. 

' .National Institutefor Education. The claringhouses 
. collect, index and abstract research reports/c^nference 
papers, curriculum materialsand handbooks, including 
many from Canadian sources, and announce them 
monthly in Hesoumin Education (HIE). They^so 
index more than 700 journals, including 26 from' 
Canada, for the Ctjmt Imjex to huwdlsin Educstion , 
, ICIJEl In all, the ERlCdata base includes over 
200,000 references, to which almost 2,500 new 
referencesaffadded each month. 

How can EISO halp? 

By searching the ERIC data base for information on 
a particular problem or area of interest, you can learn, 
hovMthers have approached the issue, E ISO provides 
direct access to ERIC, retrieving references quickly . 

. and inexpensively. Computerized searcheslof 
Dissertation Abstracts, ^ceptional Child Education 
Abstracts, Psychological. Abstracts, Social Sciences ' 
Citation Index and biological Abstracts are also 

, available. The service will be of particular lielp to' 
those working under the pressure of deadlines. For 
public addresses andYesearch reports EISO will prove 
helpful; for planning and decision.nfiaking it may 
prove indispensable. 



HowdoesEISOwork? 

A search request is submitted to the searcbanalysti a 
specially trained reference librarian in the OlSE 

' Library, who rewords the conterit to conform to a 
list of subject headings designed for computer ^ 
scanning of the data base, Via a computer terminalr^ 

' the search analyst queries the data base. When / 
combinations of subjifc headfoare entered (e.g. J 

^ remedial reading Al(lD teacher aides), the number of j 
relevant citations and sam/e citations are prii/ted \ ^ 
on-line. If the citations are appropriate," the complete 

^ set of references and abstracts are printed off-line 
and mailed to the requestor. . , f 

How long does a search take? ^ ^ 

The computer search itself normally takes fifteen] • 
■ minutes, but follows a half-hour's discussion with ihe 
search analyst whenever possible. The bibliography 
usuall|^arrives within tea working days of receipt of 
a searft request. . ; 

How much does a search cost? 
Depending on the complexity of the topic and length 
of bibliography, a search can 'cost as little as $10 or 
as much as $100, Qients are charged Sl^ per hour for 
the sear^ analyst's time (to the nearest quarter hour), 
for computer connect time, and for citations printed. 
Costs for connect time and citations vary wjth the 
data base and computer system used Contact the 
EISO search analyst for current rates, and an estimate 
of the cost for searching yourtopia 



/ How do I obtain documents and irtides? * ^ 

E'lSO provides microfiche cbpieJof all ERICdocu- / 
/ ments at a charge of 50^ per f iche, and paper copies 
of those with ED numbers'greater than ED 065 7J! 
at a charge]pf 6^ per page. Paper copies of documents 
with numbers between ED 000 001 and ED 065 734 
inclusive are^available from ERIC Document Repi^o- 
duction Service (EDRS) in the United States. Paper ' 
• copiesof journal articlesareavailablefrom ElSOata 
charge of M per pagei-Order forms for both docu- 
ments aiJj articles will bfe-sent to clients with the / ■ 
EISO bibliography,^ ■ 

How do I request ajsearcb? 

Write alehff dwcfibing the information yoti need . 
- and nwil it mhe 3ddress,below; phone 
search analysts (4.16) 923^1 ext 487; or malie 
; jn appointment\with the search ar^yst at the 
ElSO'Search Sertice Office located in *e OlSE 
Library on the second floor of the OlSE building. 

, Current av^ness service - , 

If you n^ regulai: updates on your search topic, ask ' 
the seared analyst about EISO's new current awareness 
servictf Charges will vary with the complexity of yolir 
search profile, . 

Hoft can clients assist EISO?' i< 
Besiijes-providinga useful service, EISO is also a \ 
research project looking into the information needs 
of Ontario's educators, tt will be greatly appreciated * 
if clients complete evaluation questionnaires or . 
agree to be interviewed, if jsked to do sj * ^ 

•Projeci'funded under contract with the Ontario Minijtry 
of Education. ^ 

EISO Search* Analyst ^ \ ^' • 
01S£Urary,S218 . . > 
252 Bloor Street West 
Toronto; Ontario M5S1V6 




1 
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_ APPEND'IX I 
EISO POSTER AND FLYER 



Er|c r. ' ^13 




I 



I 




onsider; for instance, the time it 
jakes to search through indexes ; 
me Resources in Education or 

t>ycho!ogical Abstracts forall the 
ferences about the topic you 
'e working on. f 



A computer can do that searching 
P less than one-tenth of the time 
|would take you to do it manually 

Is the 90% time saving worth' 
J30 to you or- to the organization 
sponsoring your work? If it is, 
jDntact: 

tiSOSekch Analyst 
ISE Library S 218 

fi2 Bipor Street West . 
ronto, Ontario M5S 1V6 
|hone:(416) 923-6641 Ext487 

I o ■ y ■■ ■ ■ — 

ERIC 



CLf AM«C*Oin< ACC MI 

otacMi^rom 
OfiCMvroAS 



cif »**'«CMOirt(ACc NO 



pu«lcatio» o*rt 
iM3TTTvrno«>MC H*^t 
OE3C*)»nvS utOTC 

0»C*<#rOMS 

A«SriMCT 



r«wo»n. juorm 

SC**O0W "tAQ*NC MOGfUH 
•VUvf Nr*»v SCHOOLS 

. -»t»V>*t*»CL3<LKCTi0n 



OOCtilMBi fUltTMC* 

*AC«.irArs A ruro«i*t m»oc««<« vmith 

ac-socACKCMLOAflNO^LOwaoooftcoNOMic t ■ 

STiruS f*HAL ■t*>OAT 
MS »3 

•'t?3*UC • > . 

ViCO COU*«TV 3CM0Ol.COM* , TIUMC MAUTL IMa 
(9^ Mice ««» flO M w&U » 

»• ^. 
'O^aOvan'aC^O tOUTm 

»t»cf "tum. uorot LeAMwic 

MniAllT COUCAnoi* 

»»0C«AJiiCU»Of5 . ■ 

'MCWtONU. IICAOwtG M0Q<UW1 

•njro*»*i. »noQ«A*»» • •-- ' > 

mw-i»c in uu w fMc necuon vxool *mx»A^. «*■ fv^a^rKjiv rSli^ 



777e L/se of Teacher Aides 
in Remedial Reading 
. Partial Printout 
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'APPENDIX :K . 
EISO SEARCH REQUEST FORM 2 



ERIC 
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EDUCATIONAL t^PORMATION SYSTEM FOR ONTARIO 

^ The Ontario Institute for Studies in Education 

Room- S218. 252 Bloor St. Wf, Toronto M5S 1V6 
W 1 6) 923-664 1 Ext 487 



SEARCH REQUEST FORM 



Name. 



Search No. 
Topic; - 



'•H. I -I ; 



Invoica: ( ) Same, or 



2.' Address 



Phbnels) { 



( 



.3. D Phone when printout arrives 
□ Mail printout without phoning 

4- Detailed description of topic 



Phone ( ) 



5. Reasons for requesting 3«arch (Pleasesjaeafy how you hopeto use the information) 



6. Citations of relevant materials known to you, if^any, or authors writing in the 



area 



OataBasejs): 



8.: Comments: 



O ERIC 
O Dii5. Abs. 
□ Other: ' 



□ Psych. Abs.* 

□ ERIC + PA 



OSSCI □ ECEA 

O ERIC + ECEA □ EQEA + PA 



ERIC 
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APPENDIX L 
"filSp SERVICE EVALUATION DATA SHEET 2 
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■ EDUCATIONAL 'INFORMATION SYSTEM *Fo\^ ONTARIO 

* • ■ 

EI SO Service Evaluation Data-Sheet 



Search No. 
Topic: 



1. 

2. 



Date (day, month, year): > » , 

.. . *■ ." ' 

Purpose : , - 



j — X__L. 



(01) 

(02) " 

(03) ' 

(04) ' 

(05) ' 

(06) ' 



• . . X . ■ ' ■ , " 
, Keep abreast of field 

Assignment, tern paper, thesis 

• Prep . , update of bibjto. 
C urriculum deve-lopment 
Program improvement ' 
P/epNspeech . article, report 



(07) 

(08) ' 

(09) " 

(10) ' 

(11) ' 



_Research/,deve lopment report 
_Brbwsing »' 
^Personnel recrait, eval., sup, 
_Ppl icy 'development 
Other: =< 



3. 
4. 
5, 
6. 



Contact : (1 ) W a t k- i h 
Payment: (1 )j 



(2) Phone 



(3) Letter 



(4) .Other: 



_Free (2)_; Cash, cheque, invoice (3) ' O lSE chargeback 

Interview, strategy time ^ t » t f"' n. • 
Requester's organization: v 



■ (1) School board (4) 

(2) CAAT (5)' 

(3) F aculty of Education (6)' 



_QISE V 

Ministry of Education 
"other: 



7. 



Please indicate your primary professional role or function. 
(Check one only) 



(01) ^Adminl strati on or supervision 

(02) T eaching 

(03) Pupil Personnel Services 

(04) Research ^ ' 

(05) ^ield Development ■-" ' 

(06) ^ ^Ministry Reg iona I Of f ice^ 

(07) L ibrary Services 



(08) Private Cionsultant 

(09) ^_ Undergraduate Student 

(10) M.Ed. Student 

(11) ^^ Ed.D. Student 

C12) M.A. Student 

(130^ Ph.D. Stud^ni" 

(14) 0th er ' 



8. Organization location: 

; ( 1 ) / M etro Torontb - ^ - : V ^ - 

(2) H am i t-ton^ London J Ottawa ^ V/i ndsor, Sudbury ^ - 

(3) D istrict or District MunlcipaJ ity. County or Regional Municipal ity 

(4) Canada (outside Ontario) 

(5) Outside Canada / . 



Sex: (1) Male 



<2) Female 



• . - "- APPENDIX M ' • ■ 

EISO USER EVALUATION -QUESTIONNAIRE 2 



y 
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i^j^^t Educational InforrhaciorL^ystem for Ontario 
■^Si^ OiSE Library, S2J8 

, 252 Bloor Street West . 
^5f?? Toronto^ Ontario M5S 1V6f 



Search No. t \ ^ \ \ \ 
Topic: ^' 



EISO. User Evaluation Questionnaire 

The data collected from the user evaluation questionnaire wUI be used to improve the ^arch service, ai?d wiTi be treated 
in*a confidential manner. 

Please read each question carefuHy and indicate your response with a check mark: - Leave blank if the item is 
not applicable. ^ * . ' . 



Satisfaction 




!• Convenience of arrangements to obtain EISO searbtoes. 



2^ Helpfulness of search analyst or Educational Information 
Consultant. 

3. Time taken to deliver the EtSO bibliography. 
■ ■ ■ ^ - 

4, Time taken to deliver microfiche or paper copies 
ordered from EISO« 



5. Value of ^bibliography itself . ^ 
S, Value of materials located via EfSO bibir6gr^hy> ' 



7^N^ce of the search and bibliography, . • . ; 

8, How much of this relevant information retrieved was new to ybu? ^ - 
(1) _L0% X2) - I'^Q^ (3) ' !P i-SQ% 



Low 




Medium 




High 


1 


L_ 




• 


1 








r 


- 1 


1 


f 




r 


I 


1 


1 






» 


1 


r 






1 




■ r 






^ f 




■■ r 




■ i" - • ■_ 








(2j . 


(3) 




(4) ' 




more than 50% f 





9. . For hovy many of the references on your printout did you read the original' material? . 

(1) None (2) ___1-5 (3) 6>1Q ' (4) 1 1-20 (5) . 



.21 or more 



10. How did you first learn about the Educational Information System for Ontario (EISO);and its servrces? (Check 

■ ' one only,)''. ■ V; 



{!) 
(2) 
(3) 
(4) 



-Brochure or flyer 
.Advertisement 
.Prof. Dev. Day 
.Colleagues 



(6) 
(7) 
(8) 
(9) 



.Library or Librarian 
.EISO. staff member 
.Cannot recall ^ 
.Other: - 



11. On the reverse side, please comment on both the good and bad/elements of your EISO search^ Any suggestions you 
make towards improving the service are welcome, ' 



THANK YOU FOR YOUR T4ME AND COOPERATION 



ERIC 
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